Dump from Roots

Back to Contents
  • Overview
  • Format Explanation
  • Changing Size & Depth Thresholds
  • Example
  • Sorting
  • Auto total-size Threshold
  • Settings

  • Overview

    The 'dump from roots' command produces a formatted text representation of the heap, it starts from each root and then shows its children. This is similar to how the heap is traversed when being processed. To do this use the 'd' command (note that this will automatically process the dump if not already processed in memory).

    This representation below is similar to the graph (hddoc.png) in Definitions & Algorithms. Notice the indentation corresponds to depth in the tree and the left-hand column represents the depth of each object. That is <0> for roots K and A.

    <0> [48,000] 0x200000b0 [40,000] K
    <1>   [8,000] 0x200000c0 [8,000] L
    <0> [26,600] 0x50001010 [6,000] A
    <1>   [20,600] 0x50002020 [4,000] B
    <2>     [13,672] 0x50004050 [1,104] C
    <3>       [12,568] 0x50005060 [1,032] D
    <4>         [11,536] 0x50003090 [504] E
    <5>           [11,000] 0x50003070 [1,000] F
    <6>             [10,000] 0x30000080 [10,000] G
    <5>           [32] 0xe00090a0 [32] H
    <2>     [1,904] 0x50006040 [1,904] I
    <2>     [1,024] 0x10007030 [1,024] J
    

    n.b. A real dump would have names like 'java/lang/String', 'array of Object' instead of 'A', 'B' etc.

    Top


    Format Explanation

    Now for an example of actually running the command:

    > d
    
    Enter total-size threshold [1048576]
    > 0
    Enter max depth or -ve for unlimited [8]
    >
    Enter 0x to dump from one address or any value for all roots [-]
    >
    Dumping object(s)  : 2 roots, sorted by total-size
    Filter             : depth at most 8
    Prune              : N
    
    <0> [48,000] 0x200000b0 [40,000] K
    <1>   [8,000] 0x200000c0 [8,000] L
    <2>     {48,000} 0x200000b0 [40,000] K
    <1>   <20,600 root:0x50001010> 0x50002020 [4,000] B
    <0> [26,600] 0x50001010 [6,000] A
    <1>   [20,600] 0x50002020 [4,000] B
    <2>     [13,672] 0x50004050 [1,104] C
    <3>       [12,568] 0x50005060 [1,032] D
    <4>         [11,536] 0x50003090 [504] E
    <5>           [11,000] 0x50003070 [1,000] F
    <6>             [10,000] 0x30000080 [10,000] G
    <6>             {13,672} 0x50004050 [1,104] C
    <5>           [32] 0xe00090a0 [32] H
    <4>         {12,568} 0x50005060 [1,032] D
    <2>     [1,904] 0x50006040 [1,904] I
    <3>       <1,024 parent:0x50002020> 0x10007030 [1,024] J
    <2>     [1,024] 0x10007030 [1,024] J
    
    2/2 roots were dumped. There were 12 objects expanded.
    

    The line for object A shows that it has address 0x50001010, a subtree/total size of [26,600] bytes. Note that B is a child of, but not owned by K.(since B's root-owner is in fact A).

    For any node, the biggest child objects (in total size) are shown first. B has children C, I & J - this is in order of total-size and also in order of display. This also applies at the root level, e.g. K is shown before A at level <0>. See Sorting below.

    It is intended to be able to add up the sizes in square '[]' brackets to form the total-sizes:

    The total-size for K [48,000] is the size of K [40,000] plus the total-size of L (the only object it is the parent-owner for ) [8,000]. The total-size of L is the same as the size here since L doesn't own anything.

    When B is listed under K its subtree size is shown in angle-brackets <20,600 root:0x50001010> to reflect that its size does not count towards the total-size of K.

    The 2nd line for 'K' is shown in curly '{}' brackets because it has already been shown in the tree descending from 'K'. At this point, the object K is not explored any further to avoid carrying on in the K -> L -> K cycle for ever.

    Each object only has its size counted once (against its parent-owner's total-size). Object J's parent is B and so its size is listed in []'s when shown as a child of B. But when it's shown under I, it is listed as <1,024 parent:0x50002020>. This is so that you don't think the size of J [1,024] counts towards the total-size of I.

    Top


    Changing Size & Depth Thresholds

    In the output below, I have filtered on objects at least 10,000 bytes in size and only down to 4 levels deep.

    Note that objects H, I, J, & L are too small in subtree size to be shown. Objects F, G & H would be too deep in the output to display. Object H is both too small and too deep down the tree. Compare this output to that above.

    > d
    
    Enter total-size threshold [0]
    > 10000                   # only objects with total-size at least 10,000
    Enter max depth or -ve for unlimited [8]
    > 4                       # only objects with a depth up to <4>
    Enter 0x to dump from one address or any value for all roots [-]
    >
    Dumping object(s)  : 2 roots, sorted by total-size
    Filter             : total-size >= 10,000 and depth at most 4
    Prune              : N
    
    <0> [48,000] 0x200000b0 [40,000] K
    <1>   <20,600 root:0x50001010> 0x50002020 [4,000] B
          - 1 child of 0x200000b0 filtered.    # object L is too small
    <0> [26,600] 0x50001010 [6,000] A
    <1>   [20,600] 0x50002020 [4,000] B
    <2>     [13,672] 0x50004050 [1,104] C
    <3>       [12,568] 0x50005060 [1,032] D
    <4>         [11,536] 0x50003090 [504] E
                  - 2 children of 0x50003090 too deep.  # object E is at the maximum-depth
    <4>         {12,568} 0x50005060 [1,032] D
            - 2 children of 0x50002020 filtered.  # I and J are too small, B is not
    
    2/2 roots were dumped. There were 6 objects expanded.
    

    You can remove the explanations of filtering to unclutter the output, see Settings.

    I expect you realised I've made up the above example graph. Now onto a real example...

    Top


    Example

    After loading this (large) dump, I processed it with 'p' and then used the 'Objects' display to show just roots (both artificial and pure) in order of size. The commands used are in the end of the objects table documentation.

      Addr.      Size         Root-Owner   Total-Size    Descend.   Name
    ---------------------------------------------------------------------------------
    R 0x00555a30 248          -            70,907,808    837,793    class java/lang/System
    A 0x101963d0 32           -            8,380,272     43,501     java/lang/ref/Finalizer
    A 0x1d521820 112          -            3,708,912     5,975      org/apache/xpath/XPathContext
    

    The table shows the 3 biggest objects in terms of their total sizes. The total sizes of the roots are calculated by adding up the sizes of every object owned by each root. Every object which isn't a root is assigned to be owned by exactly one root object. The descendants of a root is the number of objects owned by it. See Definitions.

    It will normally be the case that most objects are referenced by many other objects indirectly and so HeapRoots presents a gross simplification of the relationships within the heap. In practice however, this seems to help because it reduces the vast complexity of even an average application's heap.

    Taking the biggest root 'class java/lang/System' (which itself only takes 248 bytes), it 'owns' 837,793 objects with a total size of around 70mb. It is a 'pure root' because nothing else references it in the heap. Many other objects (some roots, some not) could probably reach 70mb of objects in the heap, some probably could reach more. The reason this root has been chosen to own so many of the objects is probably because of a first-come first-served basis due to its low address 0x00555a30. This is quite a good strategy since classes and objects like java/lang/System often end up with low addresses (not guaranteed by any means) and are good starting points for graph traversal.

    It is also quite common to see large amounts of space used by 'java/lang/ref/Finalizer' in heapdumps. This could be normal behaviour or could indicate some kind of build up of resource which cannot be freed. You can use the types table to count finalizers. Large use of finalizers (even when resources are freed correctly in Java code) will cause severe performance problems. This can be in terms of using heap space or slowing down the VM by causing a disproportionate amount of work for GC since they require special handling. See Links page for resources on finalizers.

    The third line lists some Xalan/Xerces objects (XSL&XML handling stuff) which turned out to be causing the performance problem in a WebSphere environment. It was in fact a finalizer related issue.

    Now that I've seen the top few root objects are all over 3mb, I'll now do a 'dump from roots'. Picking a threshold value is tricky because if you set it to small, then you get so much data you just have to quit and start again. It's generally better to start conservative and iterate towards a finer grain. Also see the Auto total-size Threshold section below.

    So I now run 'd'.

    > d
    
    Enter total-size threshold [1048576]
    > 3000000       3mb
    Enter max depth or -ve for unlimited [-1]
    > 4             easy tiger, don't want too much data
    Enter 0x to dump from one address or any value for all roots [-]
    >
    Dumping object(s)  : 1055309 roots, sorted by total-size
    Filter             : total-size >= 3,000,000 and depth at most 4
    Prune              : N
    
    <0> [70,907,808] 0x00555a30 [248] class java/lang/System
    <1>   [70,864,592] 0x10919f58 [24] com/ibm/ejs/security/SecurityManager
    <2>     [70,864,568] 0x101819d8 [72] java/lang/Thread
    <3>       [70,863,592] 0x1018d2b8 [56] java/lang/ThreadGroup
    <4>         [70,862,816] 0x1018d360 [56] java/lang/ThreadGroup
                  - 3 children of 0x1018d360 too deep.
                - 3 children of 0x1018d2b8 filtered.
    <3>       <70,287,480 parent:0x1017d728> 0x101816a0 [112] com/ibm/ws/bootstrap/ExtClassLoader
              - 4 children of 0x101819d8 filtered.
          - 4 children of 0x00555a30 filtered.
    <0> [8,380,272] 0x101963d0 [32] java/lang/ref/Finalizer
    <1>   [8,380,176] 0x101ba2e8 [32] java/lang/ref/Finalizer
    <2>     [8,380,144] 0x101b8988 [32] java/lang/ref/Finalizer
    <3>       [8,380,112] 0x101b7ab8 [32] java/lang/ref/Finalizer
    <4>         [8,380,080] 0x101c0a40 [32] java/lang/ref/Finalizer
                  - 4 children of 0x101c0a40 too deep.
    <4>         {8,380,144} 0x101b8988 [32] java/lang/ref/Finalizer
                - 2 children of 0x101b7ab8 filtered.
    <3>       {8,380,176} 0x101ba2e8 [32] java/lang/ref/Finalizer
              - 2 children of 0x101b8988 filtered.
    <2>     {8,380,272} 0x101963d0 [32] java/lang/ref/Finalizer
            - 2 children of 0x101ba2e8 filtered.
          - 3 children of 0x101963d0 filtered.
    <0> [3,708,912] 0x1d521820 [112] org/apache/xpath/XPathContext
    <1>   [3,561,040] 0x1d539908 [32] org/apache/xpath/VariableStack
    <2>     [3,544,608] 0x1d54f560 [32,784] array of org/apache/xpath/objects/XObject
    <3>       [3,494,736] 0x1ddc4c10 [24] org/apache/xpath/objects/XNodeSet
    <4>         [3,494,712] 0x1ddc4e08 [120] org/apache/xpath/axes/WalkingIteratorSorted
                  - 9 children of 0x1ddc4e08 too deep.
    <4>         <3,390,048 parent:0x1d542318> 0x1d53afc8 [24] org/apache/xml/dtm/ref/DTMManagerDefault
              - 10 children of 0x1d54f560 filtered.
            - 1 child of 0x1d539908 filtered.
          - 15 children of 0x1d521820 filtered.
    <0> [3,670,304] 0x129d2930 [24] org/apache/xpath/objects/XNodeSet
    <1>   [3,670,280] 0x129d2ac8 [120] org/apache/xpath/axes/WalkingIteratorSorted
    <2>     [3,669,776] 0x128606c8 [136] org/apache/xml/dtm/ref/dom2dtm/DOM2DTM
    <3>       [3,565,784] 0x1285b490 [24] org/apache/xml/dtm/ref/DTMManagerDefault
    <4>         [3,565,760] 0x1379fdb8 [16,400] array of org/apache/xml/dtm/DTM
                  - 145 children of 0x1379fdb8 too deep.
                - 2 children of 0x1285b490 filtered.
              - 15 children of 0x128606c8 filtered.
    <2>     {3,670,280} 0x129d2ac8 [120] org/apache/xpath/axes/WalkingIteratorSorted
            - 7 children of 0x129d2ac8 filtered.
    <1>   {3,565,784} 0x1285b490 [24] org/apache/xml/dtm/ref/DTMManagerDefault
    
    4/1055309 roots were dumped. There were 20 objects expanded.
    

    Now, lets see what's further down the finalizer chain (as if we couldn't guess).

    I'll use the value in bold 0x101c0a40 above and dump from this object.

    > d 0x101c0a40  Specify 0x101c0a40 as a starting point.
                    This is using an argument/short-cut, see 'help d'.
    Dumping object(s)  : 0x101c0a40 (parent 0x101b7ab8 root 0x101963d0), sorted by total-size
    Filter             : total-size >= 3,000,000 and depth at most 4
    Prune              : N
    
    <0> [8,380,080] 0x101c0a40 [32] java/lang/ref/Finalizer
    <1>   [8,380,048] 0x101c0898 [32] java/lang/ref/Finalizer
    <2>     [8,380,016] 0x101cfde0 [32] java/lang/ref/Finalizer
    <3>       [8,379,984] 0x101db9a0 [32] java/lang/ref/Finalizer
    <4>         [8,379,952] 0x101ddcd0 [32] java/lang/ref/Finalizer
                  - 4 children of 0x101ddcd0 too deep.
    <4>         {8,380,016} 0x101cfde0 [32] java/lang/ref/Finalizer
                - 2 children of 0x101db9a0 filtered.
    <3>       {8,380,048} 0x101c0898 [32] java/lang/ref/Finalizer
              - 2 children of 0x101cfde0 filtered.
    <2>     {8,380,080} 0x101c0a40 [32] java/lang/ref/Finalizer
            - 2 children of 0x101c0898 filtered.
    <1>   <8,380,112 parent:0x101b8988> 0x101b7ab8 [32] java/lang/ref/Finalizer
          - 2 children of 0x101c0a40 filtered.
    
    There were 5 objects expanded.
    

    We just see more finalizers!. See that the 'parent' and 'root' values given in bold correspond to the objects above 0x101c0a40 at levels <3> and <0> respectively in the original output.

    I then ran 'd 0x101ddcd0' to see deeper down the tree, but just reveal more finalizers. I'm thinking they'll never end! I try setting max-depth to 300 and redirect output to a file (so that I don't flood my display):

    > dt > deeptree.txt
    
    Redirecting command output to file 'deeptree.txt'.
    
    Enter total-size threshold [3000000]
    >
    Enter max depth or -ve for unlimited [300]
    > 300              # Increase the depth limit.
    Enter 0x to dump from one address or any value for all roots [0x101c0a40]
    > 0x101c0a40       # Same object, (4 levels from root).
    

    Looking at the output:

    D:\>type deeptree.txt     In a cmd window.
    
    Dumping object(s)  : 0x101c0a40 (parent 0x101b7ab8 root 0x101963d0), sorted by total-size
    Filter             : total-size >= 3,000,000 and depth at most 300
    Prune              : N
    
    <0> [8,380,080] 0x101c0a40 [32] java/lang/ref/Finalizer
    <1>   [8,380,048] 0x101c0898 [32] java/lang/ref/Finalizer
    <2>     [8,380,016] 0x101cfde0 [32] java/lang/ref/Finalizer
    <3>       [8,379,984] 0x101db9a0 [32] java/lang/ref/Finalizer
               .................................................
    <299>                                 [8,370,304] 0x142549d8 [32] java/lang/ref/Finalizer
    <300>                                 [8,370,272] 0x1452c7b8 [32] java/lang/ref/Finalizer
                                          - 4 children of 0x1452c7b8 too deep.
    <300>                                 {8,370,336} 0x14105c90 [32] java/lang/ref/Finalizer
                                          - 2 children of 0x142549d8 filtered.
               .................................................
    There were 301 objects expanded. 
    

    So, this chain just seems to keep on going. Luckily, HeapRoots has a 'max-depth' calculation for roots which will tell us how deep this goes.

    I change the process settings to calculate 'max-depth' against each object.

    > ? p
            ..........................
    Select data to store (more data increaes memory usage) [T]
    total/subtree sizes(T), maximum depth(M), # descendants(D)
    > MT              # Calculate max-depths & total-sizes
    

    Then I recalculate with 'p'. Now running 'dm' to sort by (and show) the max-depth values, you get:

    Dumping object(s)  : 0x101c0a40 (parent 0x101b7ab8 root 0x101963d0), sorted by max-depth
    Filter             : total-size >= 3,000,000 and depth at most 300
    Prune              : N
    
    <0> [8,380,080 m:3,066] 0x101c0a40 [32] java/lang/ref/Finalizer
               .................................................
    <299>                                 [8,370,304 m:2,767] 0x142549d8 [32] java/lang/ref/Finalizer
    <300>                                 [8,370,272 m:2,766] 0x1452c7b8 [32] java/lang/ref/Finalizer
                                          - 4 children of 0x1452c7b8 too deep.
               .................................................
    

    We can see that even, at <300> levels deep. The chain has another 2766 levels to go.

    We could have run 'om' to see the deepest objects. Here's some sample output:

    Showing objects whose type is in 'AR', sorted by max-depth.
    
      Addr       Size       Root-owner Parent     Total-size  Depth  Name
    -------------------------------------------------------------------------------
    A 0x101963d0 32         -          -          8,380,272   3,070  java/lang/ref/Finalizer
    R 0x00555a30 248        -          -          70,907,808  418    class java/lang/System
    

    Note that the root has a max-depth of 3,070 and the object 0x101c0a40 (4 levels deep) that we chose to 'dump' from has a max-depth of m:3,066. Then 300 levels deeper, the object's max-depth is reduced by 300 to m:2,766. This shows that 'max-depth' is a measure of how much further there is to go in a tree.

    Running 'dt > deepertree.txt' and searching beyond just the Finalizer objects reveals that they start to reference a lot of 'org/apache' type objects. I entered '-1' for the 'max depth' prompt to give unlimited depth and reduced the size threshold to 1000000.

    Dumping object(s)  : 1055309 roots, sorted by max-depth
    Filter             : total-size >= 1,000,000   # unlimited depth
                   ...........................
    <1231>                                 [1,357,768 m:1,835] 0x2268fb70 [32] java/lang/ref/Finalizer
    <1232>                                 [1,140,160 m:174] 0x2268fb90 [40] org/apache/xpath/objects/XRTreeFrag
    

    I can then use the single object dump to get information on what exactly references any of these objects and how they are used within the application. e.g. by using 'i 0x2268fb70' to see what it references/references it from outside this 'finalizer' chain from the root.

    I confirm what exactly is in this very deep tree by running 'ts', but only showing objects owned by this root. The following table shows that a lot of the objects in this tree are 'org/apache'.

    > ts 0x101963d0        # objects with this root-owner
                   ...........................
    Tabulating objects whose root-owner equals '0x101963d0', sorted by total-size
    
    Count     Total-size  Type Name
    -------------------------------------------------------------------------------
    6,966     5,067,504   primitive array
    26        426,400     array of org/apache/xml/dtm/DTM
    13        426,192     array of org/apache/xpath/objects/XObject
    2,689     390,912     array of [I
    13        213,200     array of org/apache/xml/utils/PrefixResolver
    13        213,200     array of org/apache/xalan/templates/ElemTemplateElement
    13        213,200     array of javax/xml/transform/SourceLocator
    

    This by no means shows what the problem was, just how to have a look around in the heap. The actual problem in this scenario was a WebSphere environment problem relating to the use of 'org/apache' code and 'finalizers'. I want to write a full tutorial in future.....

    Top


    Sorting

    You can sort by 'max-depth' or number of 'descendants' if you have this data available by changing processing settings.

    Now running 'dt' ('d' defaults to 'dt') and setting the depth limit to 1 gives :

    Dumping object(s)  : 2 roots, sorted by total-size
    Filter             : depth at most 1
    Prune              : Y
    
    <0> [48,000] 0x200000b0 [40,000] K          # K is biggest
    <1>   [8,000] 0x200000c0 [8,000] L
    <0> [26,600] 0x50001010 [6,000] A
    <1>   [20,600] 0x50002020 [4,000] B
    

    Use 'dm' to sort by 'max-depth':

    > dm              # 'm' for max-depth
    
    Enter total-size threshold [0]
    >
    Enter max depth or -ve for unlimited [1]
    >
    Enter 0x to dump from one address or any value for all roots [-]
    >
    Dumping object(s)  : 2 roots, sorted by max-depth
    Filter             : depth at most 1
    Prune              : Y
    
    <0> [26,600 m:6] 0x50001010 [6,000] A       # A has deepest chain (6 deep) from it
    <1>   [20,600 m:5] 0x50002020 [4,000] B     # B's max-depth is 5,one less than A's
    <0> [48,000 m:1] 0x200000b0 [40,000] K
    <1>   [8,000 m:0] 0x200000c0 [8,000] L
    

    Above, A (with the deepest tree) is now shown first rather than K (the biggest). Note that the max-depth values are shown [26,600 m:6], here 6.

    Now running 'dd', we sort by # of 'descendants'. Here the order is the same as 'dm', but different data is shown.

    Dumping object(s)  : 2 roots, sorted by descendants
    Filter             : depth at most 1
    Prune              : Y
    
    <0> [26,600 d:9] 0x50001010 [6,000] A      # A has more descendants than K
    <1>   [20,600 d:8] 0x50002020 [4,000] B      9                           1
    <0> [48,000 d:1] 0x200000b0 [40,000] K
    <1>   [8,000 d:0] 0x200000c0 [8,000] L
    
    There are 12 objects in this dump: A & K (the roots) and their descendants (9 and 1 in quantity respectively).

    Top


    Auto total-size Threshold

    The 'd auto NNN' command automatically calculates a threshold to show about NNN objects when starting from the roots. In most cases, the number you actually see will be very close to the number specified.

    Note that 'd auto' uses a default value of 100.

    > d auto 5        # Want to show about 5 objects
    
    Setting threshold automatically to show about 5 objects.
    
    Dumping object(s)  : 2 roots, sorted by total-size
    Filter             : total-size >= 12,568
    Prune              : N
    
    <0> [48,000] 0x200000b0 [40,000] K
    <1>   <20,600 root:0x50001010> 0x50002020 [4,000] B
          - 1 child of 0x200000b0 filtered.
    <0> [26,600] 0x50001010 [6,000] A
    <1>   [20,600] 0x50002020 [4,000] B
    <2>     [13,672] 0x50004050 [1,104] C
    <3>       [12,568] 0x50005060 [1,032] D
    <4>         {12,568} 0x50005060 [1,032] D
                - 1 child of 0x50005060 filtered.
            - 2 children of 0x50002020 filtered.
    
    2/2 roots were dumped. There were 5 objects expanded.
    

    This is used by the batch option.

    Top


    Settings

    The first two settings let you unclutter the output.

    See the overview for an example of changing these values from their defaults. See the threshold section for an example of the default 'cluttered' display.

    > ? d                   # Modify settings for 'd' command
                            # Defaults in []'s :
    Settings for 'Dump from Roots' :
    
    Prune indirect references/children ? [N]
    >
    Inform about filtered objects ? [Y]
    >
    Maximum indent [32]      # If indentation was unlimited, you would get
    >                        # some very wide lines.
    

    Top