You can have a play by using the artificially constructed (in doc folder) heap dump hddocaddr.txt. Note that this file corresponds to hddoc.png.
Real dumps would have more than 12 objects with names like 'java/util/Date' instead of 'A', 'B', 'C' etc. Also, the names in the dumps wouldn't be unique (e.g. there could be 100,000 lines of 'java/lang/String' objects).
To get going, (from the directory, D:\HR205 in this case, where you unzipped the .zip), run:
D:\HR207>java -jar HR207.jar doc\hddocaddr.txt HeapRoots version 2.0.7 Copyright (c) IBM Corporation 2003. All rights reserved. Opening 'doc\hddocaddr.txt' as a 'Heapdump text file' ... .............. # omitted for now. Enter: o{a,s,t,d,m,n}, g{c,s}, t{c,s,n}, i, p, d{t,d,m} or help for more info >
You now get a prompt Enter: waiting for a command. (See the section below on loading and the help system).
Enter 'os 1-5' and hit return:
> os 1-5 # This lists the 5 biggest objects Showing all objects, sorted by size. Addr Size Name ------------------------------------------------------------------------------- 0x200000b0 40,000 K 0x30000080 10,000 G 0x200000c0 8,000 L 0x50001010 6,000 A 0x50002020 4,000 B
This shows you that the biggest object is 'K' and uses 40,000 bytes of heap space. Have a look at how this corresponds to the heapdump file hddocaddr.txt.
Please read Definitions & Algorithms where you will find an illustration of this example heap dump file.
Top Running the .jar with no arguments gives the list of options available:-
Now loading a heapdump file hd.txt into HeapRoots gives:
The "Total object range" is the size in bytes from the start of the first object to the
end of the last object. "Heap usage" is the sum of all of the objects in the heap.
The flags are special attributes that some dumps have. For dumps with flags,
they can be seen and searched on in the
objects command.
The built-in help system is more comprehensive than this documentation since
most of it is dynamically generated based on the commands available.
Entering 'help' gives something similar to :
Here I've left out most of the commands. The top line describes the 'o/Objects' feature
which is used for querying what objects are in the dump.
One useful command is
the '?' command which changes settings of other commands e.g. '? o' will
change application settings (e.g. display/formatting options) related to the
'o/Objects' command.
The help command can get extra information about commands, e.g.
Another good command is 'x' for which I provide lots of alternatives, CTRL-C is
one of them!
Top The slowest part of using HeapRoots is usually the initial loading and parsing of the dump.
This can be alleviated by saving a compressed dump file (.sgz) to disk. To do this use
the 'save' command after loading the dump.
This file contains all of the data that is in the heapdump text file and before compression will be
approximately the same size as the memory needed by HeapRoots to load the dump. The file
helps if you want to restart HeapRoots - e.g. to increase the memory available with -Xmx.
You then don't really need the original dump - but updated versions of HeapRoots probably won't
read the state file - see below. You can write the heap dump (well, a functionally identical replica)
back to disk from HeapRoots with either
the 'save-txt' or 'save-txtgz' command.
HeapRoots can also load a GZipped heapdump text file (.gz/.txt.gz) to allow the dump to be zipped before
transport and then read without unzipping on the machine using HeapRoots.
(Use the shell command gzip <dump filename> to make a .gz file)
When given a filename, HeapRoots automatically trims any extensions and searches for files in the
following order: (run HR with no arguments for extra information)
Running / Built-in Help
D:\>java -jar <JARNAME> # where JARNAME is the name of the HeapRoots .jar file
D:\>java -jar <JARNAME> hd.txt # This loads HeapRoots
HeapRoots version xxx. Copyright (c) IBM Corporation 2003. All rights reserved.
Opening 'hd.txt' as a 'Heapdump text file' ...
Analysing file 'hd.txt' generated by Java VM 'IBM 1.3.1'
Counting Objects ...
... done 262,144
... done 524,288
... done 786,432
... done 1,048,576 Displays approx. memory required
Requesting at least 49 mb of heapspace to load heapdump (string data extra)
Allocating Objects ...
...................................................................... done.
Sorting objects by address ...
.................................... done.
Mapping refs from addresses to indexes ...
...................................................................... done.
# Unresolved references : 98. Rerun with -v option to view prompt to use verbose mode if interested
Temporarily requesting 10 mb of heapspace to reduce storage redundancy.
Sorting refs ...
......................................................................
. done.
Displays info on heapdump file
Comments : // Version: J2RE 1.3.1 IBM Windows 32 build cn131-20020923
// EOF: //
Dump has flags : false some dump formats can store extra flags/meta-data
# Objects : 2,045,571 # objects in dump
# Refs : 2,635,243 # references in dump
# Unresolved refs : 98 # references to addresses for which no object exists
Heap usage : 129,581,536 total size in bytes of all objects
Total object range : 188,203,216 total expanse in bytes of objects
Extra stats : unavailable before processing
HR Memory Usage : 41/88 mb indication of HR's memory usage (within current heap size)
Enter: o{a,s,t,d,m,n}, g{c,s}, t{c,s,n}, i, p, d{t,d,m} or help for more info
> prompt for input, arrow keys can go thru' history as per shell
Help
> help
Help Contents:
o/oa/os/ot/od/om/on ?* Searching and Listing of objects
............. more stuff .............
? {cmd} Commands marked '?' have settings which can be changed.
Run '?' on its own to modify general settings
............. more stuff .............
help {cmd} Provides help on available features.
'*' - If a command is marked with a '*' then you can
run 'help
> help o
Help on 'Objects Dump'
----------------------
............. more stuff .............
Loading/Saving compressed state of a dump
Example filename | Example size | Description | Comments |
---|---|---|---|
heapdump.sgz | 160 mb | HeapRoots compressed dump file | Most compressed format, very quick to load. The uncompressed size is the basic memory footprint of HeapRoots. Can create a replica of heapdump from this. |
heapdump.gz | 168 mb | compressed heapdump text file | Very compressed but still maintains original file which can be streamed out if necessary. |
heapdump.txt | 1128 mb | heapdump text file | Original file which can be easily be read using any (I would not recommend notepad or wordpad!) editor/viewer. |
You may see an error like :
Found compressed saved state 'hd.txt.sgz' Could not load file state 'hd.txt.sgz' :java.io.InvalidClassException: HRState; Local class not compatible: stream classdesc serialVersionUID=201 local class serialVersionUID=202 Possible cause is mismatch of HeapRoots versions. Yep Delete statefile 'hd.txt.sgz' ? [no] > nIn this case, the .sgz file was created on an older version (2.0.1) (note the serialVersionUIDs are the HR versions for HR201 onwards) of HeapRoots than the one running.
If you don't have the original dump, you can use the 'save-txt' or 'save-txtgz' command to create a replica (where available).
Top HeapRoots takes great care in not using too much heap space. During parsing of the dump, it
optimises the storage (temporarily using an increased amount of space, see below) to give a smaller footprint
once loaded (and for the compressed state file).
Once loaded, the memory usage to hold the dump data is shown:
If you are parsing large dumps (e.g. millions of objects) then you may find
you run out of memory. You may then find that if you specify a large -Xmx
value (to give the JVM more memory) that performance disintegrates. This
is because the JVM is using more memory than there is physical RAM and
garbage collection then causes huge amounts of paging. So there is a balance
between performance (-Xmx well within your physical RAM) and likelihood
of loading the dump (large -Xmx). See sub-section below.
For any sizeable dump I would recommend (as a first attempt):
All of the HR commands which need more memory allocate it dynamically. e.g. for processing the dump:
You can see more memory is now used to enable the 'dump from roots' feature. The extra data stored
(e.g. object type (root,non-root) subtree sizes & descendants numbers) is then available in other
modes such as 'objects dump' appearing as extra columns.
Features such as the gaps table, types table, object table all use some memory. The gaps and types
tables are held for quick retrieval. The object table holds a 'sort index' to make it fast to iterate
through the dump in any order (e.g. by size,name,size,descendants).
The 'clear' function removes any unnecesary data to leave just the basic dump. This lets
you recover memory for the needs of specific commands.
For example take a 10m object dump with (guessing) 15m references. This is the last line of the dump:
You will need a -Xmx value of around (also add 50mb for breathing space):-
So I ran HeapRoots with -Xmx370m on this dump (which in fact had 13m refs) and it took
285mb to load, 197mb thereafter and then 315mb in total to process.
Top You will see many prompts when HeapRoots is waiting for your input like:
The defaults are shown in []'s and will be used if you simply hit return.
Here L,M,U stand for Middle, Lower and Upper. Entering a single value M, e.g. 3 will just show line
number 3. Entering an open ended range, such as -U e.g. -3 will display lines upto -3, 9- will display
from line 9 onwards. This syntax also applies to specifying addresses for prompts like:
e.g. 0xAB-0xDE will match objects between 0xAB and 0xDE. 0xDE- will match objects
with address 0xDE or greater.
Top Many commands accept optional arguments.
For example, running 'ot 50-60' will repeat
the previous 'o' command but only show results from lines 50 to 60.
Some commands treat their arguments differently depending on the format.
For example, 'd 0x1234' will repeat the last 'd' command but starting from
address 0x1234. But 'd 1000' will do the same, but setting the 'size threshold'
to 1000.
Type 'help' for a list of options.
Some commands give lots of extra advice e.g. try 'help ot'
Commands which return tables of results, such as 'o' or 'i' have a
'display range'. This may be '1-25' indicating the first 25 results or
say '50-' indicating results from 50 onwards.
The 'up' and 'down' commands repeat the previous command but
'scroll' the output up or down by a 'page'. A page is based on how much data
you've previously retrieved or taken by default to be 25 lines.
For example:
Top Any command can have its output redirected to a file.
Simply add '> filename' to the end of the command, after any arguments.
You can also use '>> filename' to append to the end of a file :
(Could have run 'ts -5 > tmp' as a shortcut.)
The output is:
Tip: Use '>>' to append commands to a file. In a
shell window, use the 'tail -f filename' command (if you have it)
to view (-f = follow) the results as they appear.
Top When HeapRoots has a dump loaded (this could be from a saved state or not necessarily
from a heapdump text file), it is essentially storing the heap dump. This data can be
used to recreate a heap dump on disk.
The 'save-txt' command will prompt for a filename (assuming a default name which is
based on the dump name but with "HR" appended).
This shows the first line of the generated file. Note that this is different from
the original source file hddocaddr.txt because below
the dump is in address order whereas heap dumps aren't required to be.
This is an example of how the dump is a replica,
another being that unresolved references are discarded.
The 'save-txtgz' command GZips the file also.
Top Currently, there are no programs to take advantage of this feature.
To make HeapRoots wait on a socket to receive data, run:
You can change from the default port by running '-receive:port'.
The received dump can be saved locally using the 'save' or
'save-txt' commands.
Top Run with '-batch' on the command line and HeapRoots will run various commands
on the Heapdump without entering the normal interactive mode.
The results will be saved to a file (name generated from dump name).
This includes :
Dynamic Memory Usage
HR Memory Usage : 41/88 mb
> p
Requesting at least 25 mb of heapspace to process heapdump
.....................
HR Memory Usage : 68/88 mb Now using more memory.
Estimates on memory usage
These are approximate values for memory usage to guide setting the -Xmx value.
State objects storage cost
per 1m objectsreferences storage cost
per 1m referencesapprox total cost (objects+references)
per 1m objects
assuming 1.5 refs per objectParsing Dump 20mb 8mb 32mb
Thereafter/Storing Dump 16mb 4mb 22mb
Processing Data (optional) 13mb - -
Objects table (optional) 4mb - -
Gaps or Types table (optional) maybe 1mb in total - -
// EOF: Total: 9459874 Classes: 8247 Objects: 6880711 ObjectArrays: 832068 PrimitiveArrays: 1738848 //
Entering values in Prompts
Enter name to filter on or '-' for no filtering [-]
> # Hitting return will pick '-' which means 'no filtering'
Enter range of lines to print in format 'M','L-U','-U' or 'L-' [1-25]
# Default is to show results 1 to 25
Enter address range in format 'M','L-U','-U' or 'L-' [0x00000000-0xFFFFFFFF]
# Default here is 0x00000000-0xFFFFFFFF
Shortcuts / Saving Time
Command Arguments
Up / Down
o 1-10
down
down
is the same as
o 1-10
o 11-20 page size is assumed to be 10
o 21-30
and
o -25
down
down
is the same as
o -25
o -50 page size is default of 25
o -75
Line Width, Verbose mode
Run '?' and you can chose
Redirecting Output to a File
> ts > tmp # sending to file in current dir called 'tmp'
Redirecting command output to file 'tmp'.
Enter name to filter on or '-' for no filtering [-]
>
Enter range of lines to print in format 'M','L-U','-U' or 'L-' [1-25]
> -5
# cat tmp In a shell window, list the file created
Use 'type' on Windows or any editor
Tabulating all objects, sorted by total-size
Count Total-size Type Name
-------------------------------------------------------------------------------
615,526 81,976,688 primitive array
506,848 16,219,136 java/lang/String
196,572 6,290,304 java/util/Hashtable$Entry
172,940 4,150,560 com/ibm/ejs/security/util/Cache$Entry
84,993 4,079,664 com/ibm/WebSphereSecurity/Credential
Displayed results : -5
Matched types : 7,052 / 7,052
Usage count : 2,045,571 / 2,045,571
Total size : 129,581,536 / 129,581,536
Writing (a Replica of) the dump (back) to Disk
> save-txt # can supply filename as arg
Enter filename to write heap dump to [hddocaddrHR.txt]
> # accept default
Writing heap dump to 'hddocaddrHR.txt' ...
................................................................. done.
D:\>head hddocaddrHR.txt
// HeapRoots version xxx heapdump generated 'Tue Sep 30 12:26:23 BST 2003' from 'hddocaddr'
0x10007030 [1024] J
............... etc .............
> save-txtgz replica.gz # I have supplied the filename I want
Writing gzipped heap dump to 'replica.gz' ...
................................................................. done.
Receiving a dump across Network
D:\>java -jar <JARNAME> -receive
HeapRoots version xxx. Copyright (c) IBM Corporation 2003. All rights reserved.
Listening on port 21179 ... # Wait for data ...
Connection received from '127.0.0.1'. # Sending from the local machine
Stream input is of type 'Compresed Heapdump text file' (.txt.gz).
Allocating dynamic arrays for stream input data ...
....... more stuff ........
Dump comments : // JFormat dump generated Tue Sep 30 13:27:26 BST 2003
from 'hddocaddr.txt' # generated by JFormat
....... more stuff ........
Enter: o{a,s,t,d,m,n}, g{c,s}, t{c,s,n}, i, p, d{t,d,m} or help for more info
>
Batch Mode