User Interface

Back to Contents
  • Getting Started
  • Running / Built-in Help
  • Loading/Saving compressed state of a dump
  • Entering values in Prompts
  • Shortcuts / Saving Time
  • Dynamic Memory Usage
  • Redirecting Output to a File
  • Writing (a Replica of) the dump (back) to Disk
  • Receiving a dump across Network
  • Batch Mode

  • Getting Started

    You can have a play by using the artificially constructed (in doc folder) heap dump hddocaddr.txt. Note that this file corresponds to hddoc.png.

    Real dumps would have more than 12 objects with names like 'java/util/Date' instead of 'A', 'B', 'C' etc. Also, the names in the dumps wouldn't be unique (e.g. there could be 100,000 lines of 'java/lang/String' objects).

    To get going, (from the directory, D:\HR205 in this case, where you unzipped the .zip), run:

    D:\HR207>java -jar HR207.jar doc\hddocaddr.txt
    HeapRoots version 2.0.7 Copyright (c) IBM Corporation 2003. All rights reserved.
    
    Opening 'doc\hddocaddr.txt' as a 'Heapdump text file' ...
            .............. # omitted for now.
    Enter: o{a,s,t,d,m,n}, g{c,s}, t{c,s,n}, i, p, d{t,d,m} or help for more info
    >
    

    You now get a prompt Enter: waiting for a command. (See the section below on loading and the help system).

    Enter 'os 1-5' and hit return:

    > os 1-5               # This lists the 5 biggest objects 
    
    Showing all objects, sorted by size.
    
    Addr       Size       Name
    -------------------------------------------------------------------------------
    0x200000b0 40,000     K
    0x30000080 10,000     G
    0x200000c0 8,000      L
    0x50001010 6,000      A
    0x50002020 4,000      B
    

    This shows you that the biggest object is 'K' and uses 40,000 bytes of heap space. Have a look at how this corresponds to the heapdump file hddocaddr.txt.

    Please read Definitions & Algorithms where you will find an illustration of this example heap dump file.

    Top


    Running / Built-in Help

    Running the .jar with no arguments gives the list of options available:-

    D:\>java -jar <JARNAME>    # where JARNAME is the name of the HeapRoots .jar file
    

    Now loading a heapdump file hd.txt into HeapRoots gives:

    D:\>java -jar <JARNAME> hd.txt       # This loads HeapRoots
    HeapRoots version xxx. Copyright (c) IBM Corporation 2003. All rights reserved.
    
    Opening 'hd.txt' as a 'Heapdump text file' ...
    Analysing file 'hd.txt' generated by Java VM 'IBM 1.3.1'
    
    Counting Objects ...
       ... done 262,144
       ... done 524,288
       ... done 786,432
       ... done 1,048,576                 Displays approx. memory required
    Requesting at least 49 mb of heapspace to load heapdump (string data extra)
    Allocating Objects ...
    ...................................................................... done.
    Sorting objects by address ...
    ....................................                                   done.
    Mapping refs from addresses to indexes ...
    ...................................................................... done.
    # Unresolved references : 98. Rerun with -v option to view    prompt to use verbose mode if interested
    Temporarily requesting 10 mb of heapspace to reduce storage redundancy.
    Sorting refs ...
    ......................................................................
    .                                                                      done.
    
                                         Displays info on heapdump file
    Comments           : // Version: J2RE 1.3.1 IBM Windows 32 build cn131-20020923
                         // EOF: //
    Dump has flags     : false           some dump formats can store extra flags/meta-data
    # Objects          : 2,045,571       # objects in dump
    # Refs             : 2,635,243       # references in dump
    # Unresolved refs  : 98              # references to addresses for which no object exists
    Heap usage         : 129,581,536     total size in bytes of all objects
    Total object range : 188,203,216     total expanse in bytes of objects
    
    Extra stats        : unavailable before processing
    
    HR Memory Usage    : 41/88 mb        indication of HR's memory usage (within current heap size)
    
    Enter: o{a,s,t,d,m,n}, g{c,s}, t{c,s,n}, i, p, d{t,d,m} or help for more info
    >                                    prompt for input, arrow keys can go thru' history as per shell
    

    The "Total object range" is the size in bytes from the start of the first object to the end of the last object. "Heap usage" is the sum of all of the objects in the heap.

    The flags are special attributes that some dumps have. For dumps with flags, they can be seen and searched on in the objects command.

    Help

    The built-in help system is more comprehensive than this documentation since most of it is dynamically generated based on the commands available. Entering 'help' gives something similar to :

    > help
    
    Help Contents:
    
    o/oa/os/ot/od/om/on  ?* Searching and Listing of objects
    
    ............. more stuff .............
    
    ? {cmd}                 Commands marked '?' have settings which can be changed.
                            Run '?' on its own to modify general settings
    
    ............. more stuff .............
    
    help {cmd}              Provides help on available features.
                            '*' - If a command is marked with a '*' then you can
                            run 'help ' to get further help.
                            'cmd {arg}' means that a command has an optional
                            argument 'arg'.
    
    x/exit/quit             Exits
    

    Here I've left out most of the commands. The top line describes the 'o/Objects' feature which is used for querying what objects are in the dump.

    One useful command is the '?' command which changes settings of other commands e.g. '? o' will change application settings (e.g. display/formatting options) related to the 'o/Objects' command.

    The help command can get extra information about commands, e.g.

    > help o
    
    Help on 'Objects Dump'
    ----------------------
    
    ............. more stuff .............
    

    Another good command is 'x' for which I provide lots of alternatives, CTRL-C is one of them!

    Top


    Loading/Saving compressed state of a dump

    The slowest part of using HeapRoots is usually the initial loading and parsing of the dump. This can be alleviated by saving a compressed dump file (.sgz) to disk. To do this use the 'save' command after loading the dump.

    This file contains all of the data that is in the heapdump text file and before compression will be approximately the same size as the memory needed by HeapRoots to load the dump. The file helps if you want to restart HeapRoots - e.g. to increase the memory available with -Xmx.

    You then don't really need the original dump - but updated versions of HeapRoots probably won't read the state file - see below. You can write the heap dump (well, a functionally identical replica) back to disk from HeapRoots with either the 'save-txt' or 'save-txtgz' command.

    HeapRoots can also load a GZipped heapdump text file (.gz/.txt.gz) to allow the dump to be zipped before transport and then read without unzipping on the machine using HeapRoots. (Use the shell command gzip <dump filename> to make a .gz file)

    When given a filename, HeapRoots automatically trims any extensions and searches for files in the following order: (run HR with no arguments for extra information)

    Example filenameExample sizeDescriptionComments
    heapdump.sgz160 mb HeapRoots compressed dump file Most compressed format, very quick to load. The uncompressed size is the basic memory footprint of HeapRoots. Can create a replica of heapdump from this.
    heapdump.gz 168 mb compressed heapdump text file Very compressed but still maintains original file which can be streamed out if necessary.
    heapdump.txt 1128 mb heapdump text fileOriginal file which can be easily be read using any (I would not recommend notepad or wordpad!) editor/viewer.

    You may see an error like :

    Found compressed saved state 'hd.txt.sgz'
    Could not load file state 'hd.txt.sgz' :java.io.InvalidClassException: HRState; Local class not compatible: stream classdesc serialVersionUID=201 local class serialVersionUID=202
    Possible cause is mismatch of HeapRoots versions.     Yep
    
    Delete statefile 'hd.txt.sgz' ? [no]
    > n
    
    In this case, the .sgz file was created on an older version (2.0.1) (note the serialVersionUIDs are the HR versions for HR201 onwards) of HeapRoots than the one running.

    If you don't have the original dump, you can use the 'save-txt' or 'save-txtgz' command to create a replica (where available).

    Top


    Dynamic Memory Usage

    HeapRoots takes great care in not using too much heap space. During parsing of the dump, it optimises the storage (temporarily using an increased amount of space, see below) to give a smaller footprint once loaded (and for the compressed state file).

    Once loaded, the memory usage to hold the dump data is shown:

    HR Memory Usage    : 41/88 mb

    If you are parsing large dumps (e.g. millions of objects) then you may find you run out of memory. You may then find that if you specify a large -Xmx value (to give the JVM more memory) that performance disintegrates. This is because the JVM is using more memory than there is physical RAM and garbage collection then causes huge amounts of paging. So there is a balance between performance (-Xmx well within your physical RAM) and likelihood of loading the dump (large -Xmx). See sub-section below.

    For any sizeable dump I would recommend (as a first attempt):

  • max heap - 80% of physical RAM if you can spare it - e.g. -Xmx400m on my 512mb RAM machine
  • min heap - don't specify -Xms, it won't generally help

    All of the HR commands which need more memory allocate it dynamically. e.g. for processing the dump:

    > p
    
    Requesting at least 25 mb of heapspace to process heapdump
    .....................
    HR Memory Usage    : 68/88 mb      Now using more memory.
    

    You can see more memory is now used to enable the 'dump from roots' feature. The extra data stored (e.g. object type (root,non-root) subtree sizes & descendants numbers) is then available in other modes such as 'objects dump' appearing as extra columns.

    Features such as the gaps table, types table, object table all use some memory. The gaps and types tables are held for quick retrieval. The object table holds a 'sort index' to make it fast to iterate through the dump in any order (e.g. by size,name,size,descendants).

    The 'clear' function removes any unnecesary data to leave just the basic dump. This lets you recover memory for the needs of specific commands.

    Estimates on memory usage

    These are approximate values for memory usage to guide setting the -Xmx value.

    Stateobjects storage cost
    per 1m objects
    references storage cost
    per 1m references
    approx total cost (objects+references)
    per 1m objects
    assuming 1.5 refs per object
    Parsing Dump20mb8mb32mb
    Thereafter/Storing Dump16mb4mb22mb
    Processing Data (optional) 13mb--
    Objects table (optional) 4mb--
    Gaps or Types table (optional) maybe 1mb in total--

    For example take a 10m object dump with (guessing) 15m references. This is the last line of the dump:

    // EOF: Total: 9459874 Classes: 8247 Objects: 6880711 ObjectArrays: 832068 PrimitiveArrays: 1738848 //

    You will need a -Xmx value of around (also add 50mb for breathing space):-

  • 320m for parsing
  • 220m thereafter or reloading from a saved state
  • 350m in total for processing
  • So I ran HeapRoots with -Xmx370m on this dump (which in fact had 13m refs) and it took 285mb to load, 197mb thereafter and then 315mb in total to process.

    Top


    Entering values in Prompts

    You will see many prompts when HeapRoots is waiting for your input like:

    Enter name to filter on or '-' for no filtering [-]
    >           # Hitting return will pick '-' which means 'no filtering'
    

    The defaults are shown in []'s and will be used if you simply hit return.

    Enter range of lines to print in format 'M','L-U','-U' or 'L-' [1-25]
                # Default is to show results 1 to 25
    

    Here L,M,U stand for Middle, Lower and Upper. Entering a single value M, e.g. 3 will just show line number 3. Entering an open ended range, such as -U e.g. -3 will display lines upto -3, 9- will display from line 9 onwards. This syntax also applies to specifying addresses for prompts like:

    Enter address range in format 'M','L-U','-U' or 'L-' [0x00000000-0xFFFFFFFF]
                # Default here is 0x00000000-0xFFFFFFFF
    

    e.g. 0xAB-0xDE will match objects between 0xAB and 0xDE. 0xDE- will match objects with address 0xDE or greater.

    Top


    Shortcuts / Saving Time

    Command Arguments

    Many commands accept optional arguments.

    For example, running 'ot 50-60' will repeat the previous 'o' command but only show results from lines 50 to 60.

    Some commands treat their arguments differently depending on the format. For example, 'd 0x1234' will repeat the last 'd' command but starting from address 0x1234. But 'd 1000' will do the same, but setting the 'size threshold' to 1000.

    Type 'help' for a list of options. Some commands give lots of extra advice e.g. try 'help ot'

    Up / Down

    Commands which return tables of results, such as 'o' or 'i' have a 'display range'. This may be '1-25' indicating the first 25 results or say '50-' indicating results from 50 onwards.

    The 'up' and 'down' commands repeat the previous command but 'scroll' the output up or down by a 'page'. A page is based on how much data you've previously retrieved or taken by default to be 25 lines.

    For example:

    o 1-10
    down
    down
    
    is the same as
    o 1-10
    o 11-20     page size is assumed to be 10
    o 21-30
    
    and
    o -25
    down
    down
    
    is the same as
    o -25
    o -50       page size is default of 25
    o -75
    

    Line Width, Verbose mode

    Run '?' and you can chose
  • verbose mode - prints extra info for debugging, e.g. reporting errors
  • line-width - some commands wrap output
  • Top


    Redirecting Output to a File

    Any command can have its output redirected to a file.

    Simply add '> filename' to the end of the command, after any arguments. You can also use '>> filename' to append to the end of a file :

    > ts > tmp    # sending to file in current dir called 'tmp'
    
    Redirecting command output to file 'tmp'.
    
    Enter name to filter on or '-' for no filtering [-]
    >
    Enter range of lines to print in format 'M','L-U','-U' or 'L-' [1-25]
    > -5
    

    (Could have run 'ts -5 > tmp' as a shortcut.)

    The output is:

    # cat tmp      In a shell window, list the file created
                   Use 'type' on Windows or any editor
    Tabulating all objects, sorted by total-size
    
    Count     Total-size  Type Name
    -------------------------------------------------------------------------------
    615,526   81,976,688  primitive array
    506,848   16,219,136  java/lang/String
    196,572   6,290,304   java/util/Hashtable$Entry
    172,940   4,150,560   com/ibm/ejs/security/util/Cache$Entry
    84,993    4,079,664   com/ibm/WebSphereSecurity/Credential
    
    Displayed results : -5
    Matched types     : 7,052 / 7,052
    Usage count       : 2,045,571 / 2,045,571
    Total size        : 129,581,536 / 129,581,536
    

    Tip: Use '>>' to append commands to a file. In a shell window, use the 'tail -f filename' command (if you have it) to view (-f = follow) the results as they appear.

    Top


    Writing (a Replica of) the dump (back) to Disk

    When HeapRoots has a dump loaded (this could be from a saved state or not necessarily from a heapdump text file), it is essentially storing the heap dump. This data can be used to recreate a heap dump on disk.

    The 'save-txt' command will prompt for a filename (assuming a default name which is based on the dump name but with "HR" appended).

    > save-txt        # can supply filename as arg
    
    Enter filename to write heap dump to [hddocaddrHR.txt]
    >                 # accept default
    
    Writing heap dump to 'hddocaddrHR.txt' ...
    .................................................................      done.
    

    This shows the first line of the generated file. Note that this is different from the original source file hddocaddr.txt because below the dump is in address order whereas heap dumps aren't required to be.

    D:\>head hddocaddrHR.txt
    // HeapRoots version xxx heapdump generated 'Tue Sep 30 12:26:23 BST 2003' from 'hddocaddr'
    0x10007030 [1024] J
    ............... etc .............
    

    This is an example of how the dump is a replica, another being that unresolved references are discarded.

    The 'save-txtgz' command GZips the file also.

    > save-txtgz replica.gz    # I have supplied the filename I want
    
    
    Writing gzipped heap dump to 'replica.gz' ...
    .................................................................      done.
    

    Top


    Receiving a dump across Network

    Currently, there are no programs to take advantage of this feature.

    To make HeapRoots wait on a socket to receive data, run:

    D:\>java -jar <JARNAME> -receive
    HeapRoots version xxx. Copyright (c) IBM Corporation 2003. All rights reserved.
    
    Listening on port 21179 ...             # Wait for data ...
    Connection received from '127.0.0.1'.   # Sending from the local machine
    Stream input is of type 'Compresed Heapdump text file' (.txt.gz).
    Allocating dynamic arrays for stream input data ...
    
    ....... more stuff ........
    
    Dump comments        : // JFormat dump generated Tue Sep 30 13:27:26 BST 2003
                           from 'hddocaddr.txt'   # generated by JFormat
    
    ....... more stuff ........
    
    Enter: o{a,s,t,d,m,n}, g{c,s}, t{c,s,n}, i, p, d{t,d,m} or help for more info
    >
    

    You can change from the default port by running '-receive:port'.

    The received dump can be saved locally using the 'save' or 'save-txt' commands.

    Top


    Batch Mode

    Run with '-batch' on the command line and HeapRoots will run various commands on the Heapdump without entering the normal interactive mode.

    The results will be saved to a file (name generated from dump name).

    This includes :

  • save the state for quick retrieval - 'save'
  • most frequent gaps - 'gc'
  • biggest types - 'ts'
  • statistics - 'stats'
  • biggest total-size root objects - 'ot' with only 'A,R' types selected
  • dump from roots to show about 100 objects - 'dt auto 100'
  • dump from roots - 'dt auto 250' but with pruning on, notify off
  • Top