The default GlassFish installations has a JMX listening socket on port 8686 (for GlassFish v2ur2) or 8696 (for GlassFish v2.1-b60) that speaks some type of RMI protocol. To connect jconsole to this port, start jconsole and enter this URL:

    service:jmx:rmi:///jndi/rmi://www.example.com:8686/jmxrmi

and fill in "admin" as the username, plus the corresponding password. And change www.example.com to the GlassFish host, and change the port number to 8696 if you are running v2.1-something (I don't know when they changed the port number).

If you have firewalled this port, you can tunnel jmxrmi over ssh. You need to make ssh forward two ports: 8686 (or 8696) and a randomized port with a high port number. This unknown port number can be determined like this:

    # lsof -nPp 4711 | grep LISTEN

where you change "4711" to whatever the GlassFish pid is. The line below TCP *:8686 (LISTEN) will typically contain the second port number. If it doesn't work, try one of the other high port numbers. Then create the ssh tunnels, start jconsole, and enter localhost as the host in the jmxrmi URL.

Now you can track the memory and CPU load of GlassFish on your remote server. If the CPU load seems abnormally high, you can take a snapshot of all the threads by doing "jstack 4711" in a server shell. For some bizarre reason, jstack running as root refuses to dump stack traces of a GlassFish process that is running as some other user. So you need to do e.g. "sudo -u myglassfishuser jstack 4711" from your root shell.

[Footnote: GlassFish v2.1-b60a has moved back the rmi socket to port 8686 again]

A few "remind-myself" notes about various stumbling blocks that I encountered while writing my first Java EE code for GlassFish:

• You need $GLASSFISH_HOME/lib/javaee.jar in the classpath when compiling.
• The JMS Destination Resource must have the property Name=PhysicalQueue, or Name=PhysicalTopic for a topic.
• Resouce injection by annotations doesn't work for arbitrary classes or JSP sources. It only works for container managed components like EJBs, servlets, servlet filters, JSP tag handlers, application-client "main" classes,
• The server log is spammed with a multitude of these messaging exceptions:
  "com.sun.messaging.jmq.io.Packet cannot be cast to com.sun.messaging.jms.ra.DirectPacket"
  Follow the instructions here to have JMS run in a separate process. Note: this problem has been fixed in the GlassFish v2.1 beta.
  
• The web-app XML schema for "web.xml" must be version 2.5 for annotations to work.
• For injection to work in application clients, the injection variables must be static. And in order to inject EJBs the client must be packaged in an EAR file and declared in "application.xml".
• Injection variables cannot be static in servlets and filters.
• Persistent entity classes must be placed in jars under "lib" in the EAR file.
• Servlet and filter objects should not have EntityManagers as instance variables, since EntityManagers are not thread-safe. Instead, either use an EJB as an instance variable and let the EJB access the EntityManager, or use EntityManagerFactory to create a temporary EntityManager and manage the transactions yourself.
• Any webapp that requires login needs a "sun-web.xml" that declares <security-role-mapping>, otherwise no one is allowed access. Or if the webapp is packaged in an EAR file, the same mapping can be declared in "sun-application.xml". Also, the <realm-name> in "web.xml" cannot contain space characters.
• If you change the admin password, asadmin login is needed to update the stored credentials in ~/.asadminpass.
• Use @TableGenerator annotation for a portable way to generate primary keys for persistent @Entity objects. The generated keys are filled in by the first call to EntityManager.persist(), and the new key is committed by EntityManager.flush().
• Log Toplink-generated SQL statements by adding the following property under <persistence-unit> in "persistence.xml":
  <property name="toplink.logging.level" value="FINE"/>