1 basic knowledge
Heap and stack
When a Java program moves, the memory space can be divided into five blocks according to different functions: register, local method area, method area, heap and stack.
In terms of heap and stack alone, the stack mainly stores basic data types and local variables. The instance objects or data created by new are stored in the heap.
For the stack, each thread has its own independent stack. When the thread ends, the stack will disappear (the space will be recycled. In fact, there is no need to wait for the thread to end. As the method runs in and out of the stack, the content related to the method will disappear). The heap is shared by everyone. Instance objects or data may not disappear immediately even if they are no longer used. Because the collection of heap memory depends on the garbage collection mechanism.
Object header
Java objects are generally composed of three parts: object header, instance data and object filling.
The object header usually has Mark Word and pointer to the class. There will be an extra part of the array object to store the length of the array.
Mark Word mainly stores some basic information of the object, such as GC mark, lock information, etc. Generally, 32-bit JVMs occupy 32 bits and 64 bit JVMs occupy 64 bits.
The pointer to the Class is usually 32bit or 64bit. The function of this thing is to mark which Class the instance object belongs to. We know that Class objects will be placed in the method area when the Class is loaded. In fact, the pointer to the Class points to the address where the Class object is stored in the method area.
Array length holds the length of the array object. Only array objects have this content, which is 32bit long.
Pointer compression
In order to reduce the memory space occupied by the object, the JVM allows the object UseCompressedOops parameter (on by default) to set pointer compression on.
When pointer compression is enabled, it will mainly affect the object header and reference type. After compression is turned on, the space occupied by the object header becomes 12 bytes. If it is an array object, it will be 16 bytes (the additional 4 bytes are used to store the array length). The reference type is changed from 8 bytes to 4 bytes.
Object fill
Because the instance object is complex, let's introduce object filling first. In the JVM, the content space occupied by all objects is an integer multiple of 8bit. If it is insufficient, it should be supplemented. The supplementary content is meaningless.
Instance data
Instance data is generally the class attribute part of an object. This part of the content occupies different memory sizes according to different types.
The reference type occupies 4 bytes each on a 32-bit system and 8 bytes each on a 64 bit system (4 bytes after pointer compression).
The structure of ordinary objects is as follows, calculated according to the length of 64 bit machines
1. Object header (_mark), 8 bytes
2. If the OOP pointer is below 32G memory, object pointer compression is enabled by default, with 4 bytes
3. Data area
4. Padding: align according to the multiple of 8
The array object structure is
1. Object header (_mark), 8 bytes
2. If the OOP pointer is below 32G memory, object pointer compression is enabled by default, with 4 bytes
3. Array length, 4 bytes
4. Data area
5. Padding: align according to the multiple of 8
After knowing the basic layout of objects in memory, let's talk about two methods to calculate the size of Java objects
- Through Java lang.instrument. getObjectSize(obj) of instrumentation gets the size of the object directly
- Via sun misc. Methods such as objectFieldOffset(field) of unsafe objects combine reflection to calculate the size of objects
2 getObjectSize(obj) of instrument
Let's start with Java lang.instrument. Instrumentation. Getobjectsize(), which gets the Shallow Size, that is, when a reference is encountered, only the length of the reference is calculated, and the actual size of the referenced object is not calculated. If you want to calculate the actual size of the referenced object, you can calculate it recursively.
java. lang.instrument. The instance of instrumentation can only be obtained by specifying javaagent. The specific steps are as follows:
1. Define a class and provide a premain method: public static void premain(String agentArgs, Instrumentation instP)
2. Create meta-inf / manifest MF file, which specifies the class of PreMain: PreMain class: sizeof ObjectShallowSize
3. Type this class into jar, and then use Java - javaagent XXXX jar XXX. Main
Let's define a class to get Java lang.instrument. And provides a static sizeOf method with the ability to provide external Instrumentation
package sizeof;
import java.lang.instrument.Instrumentation;
public class ObjectShallowSize {
private static Instrumentation inst;
public static void premain(String agentArgs, Instrumentation instP){
inst = instP;
}
public static long sizeOf(Object obj){
return inst.getObjectSize(obj);
}
} Define meta-inf / manifest MF file
Premain-Class: sizeof.ObjectShallowSize
Make jar package
cd Compiled classes and META-INF Folder directory jar cvfm java-agent-sizeof.jar META-INF/MANIFEST.MF
After the jar is ready, we can write a test class to test the getObjectSize method of Instrumentation. Before that, let's look at the order in which objects are arranged in memory
There are the following classes, and the field definitions are in the following order
private static class ObjectA {
String str; // 4
int i1; // 4
byte b1; // 1
byte b2; // 1
int i2; // 4
ObjectB obj; //4
byte b3; // 1
} Calculate the size of this object according to the method we mentioned earlier. Pay attention to align by 8
8(_mark) + 4(oop pointer) + 4(str) + 4(i1) + 1(b1) + 1(b2) + 2(padding) + 4(i2) + 4(obj) + 1(b3) + 7(padding) = 40?
Actually, is that so? Let's use getObjectSize of Instrumentation to calculate. First:
package test;
import sizeof.ObjectShallowSize;
public class SizeofWithInstrumetation {
private static class ObjectA {
String str; // 4
int i1; // 4
byte b1; // 1
byte b2; // 1
int i2; // 4
ObjectB obj; //4
byte b3; // 1
}
private static class ObjectB {
}
public static void main(String[] args){
System.out.println(ObjectShallowSize.sizeOf(new ObjectA()));
}
} The result is 32! Don't you know how to align by 8? The data before b3 adds up to 32. There is one more b3, which is 33. It should be aligned to 40. In fact, the fields of objects created by HotSpot will be arranged in the given order first. The default order is as follows: long/double – > int / float – > short / char – > byte / Boolean – > reference
This order can be changed using the JVM parameter: - XX: fieldsallocationsylte = 0 (the default is 1).
We use sun misc. Verify with objectFieldOffset method of unsafe object:
Field[] fields = ObjectA.class.getDeclaredFields();
for(Field f: fields){
System.out.println(f.getName() + " offset: " +unsafe.objectFieldOffset(f));
} 
You can see that it is indeed arranged in memory from long to short and the reference is the last. In this way, let's recalculate the length of the object created by ObjectA:
8(_mark) + 4(oop pointer) + 4(i1) + + 4(i2) + 1(b1) + 1(b2) + 1(b3) + 1(padding) + 4(str) + 4(obj) = 32
The result is similar to that of Java lang.instrument. Instrumentation. The result of getobjectsize () is the same, which proves that our calculation method is correct.
3 sun.misc.Unsafe way
Let's talk about sun misc. Methods such as objectFieldOffset(field) of unsafe object are combined with reflection to calculate the size of the object. The basic idea is as follows:
1. Get the Field of a class through reflection
2. Obtain the offSet of each Field through objectFieldOffset() of Unsafe
3. Sort the fields by offset, get the maximum offset, and then add the length of the field and the Padding alignment
In the above three steps, you can obtain the Shallow size of an object. You can further calculate the size of the referenced object through recursion, so as to calculate the actual size occupied by an object.
How to get Unsafe objects has been in this article Talk about serialization (2) using sun misc. Unsafe bypasses the new mechanism to create Java objects As I said, it can be obtained through the mechanism of reflection
Whether the Oop pointer is 4 or uncompressed 8 can also be passed through unsafe Arrayindexscale (object []. Class). This method returns the length occupied by a reference
static {
try {
Field field = Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (Unsafe) field.get(null);
objectRefSize = unsafe.arrayIndexScale(Object[].class);
} catch (Exception e) {
throw new RuntimeException(e);
}
} Source code from below http://java-performance.info/memory-introspection-using-sun-misc-unsafe-and-reflection/ , the code in the original text has a problem in calculating the object size. I made a fine adjustment and added the memory alignment method. In this way, the calculated result is the same as the getObjectSize method of Instrumentation.
package test;
import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;
/**
* This class contains object info generated by ClassIntrospector tool
*/
public class ObjectInfo {
/** Field name */
public final String name;
/** Field type name */
public final String type;
/** Field data formatted as string */
public final String contents;
/** Field offset from the start of parent object */
public final int offset;
/** Memory occupied by this field */
public final int length;
/** Offset of the first cell in the array */
public final int arrayBase;
/** Size of a cell in the array */
public final int arrayElementSize;
/** Memory occupied by underlying array (shallow), if this is array type */
public final int arraySize;
/** This object fields */
public final List<ObjectInfo> children;
public ObjectInfo(String name, String type, String contents, int offset, int length, int arraySize,
int arrayBase, int arrayElementSize)
{
this.name = name;
this.type = type;
this.contents = contents;
this.offset = offset;
this.length = length;
this.arraySize = arraySize;
this.arrayBase = arrayBase;
this.arrayElementSize = arrayElementSize;
children = new ArrayList<ObjectInfo>( 1 );
}
public void addChild( final ObjectInfo info )
{
if ( info != null )
children.add( info );
}
/**
* Get the full amount of memory occupied by a given object. This value may be slightly less than
* an actual value because we don't worry about memory alignment - possible padding after the last object field.
*
* The result is equal to the last field offset + last field length + all array sizes + all child objects deep sizes
* @return Deep object size
*/
public long getDeepSize()
{
//return length + arraySize + getUnderlyingSize( arraySize != 0 );
return addPaddingSize(arraySize + getUnderlyingSize( arraySize != 0 ));
}
long size = 0;
private long getUnderlyingSize( final boolean isArray )
{
//long size = 0;
for ( final ObjectInfo child : children )
size += child.arraySize + child.getUnderlyingSize( child.arraySize != 0 );
if ( !isArray && !children.isEmpty() ){
int tempSize = children.get( children.size() - 1 ).offset + children.get( children.size() - 1 ).length;
size += addPaddingSize(tempSize);
}
return size;
}
private static final class OffsetComparator implements Comparator<ObjectInfo>
{
@Override
public int compare( final ObjectInfo o1, final ObjectInfo o2 )
{
return o1.offset - o2.offset; //safe because offsets are small non-negative numbers
}
}
//sort all children by their offset
public void sort()
{
Collections.sort( children, new OffsetComparator() );
}
@Override
public String toString() {
final StringBuilder sb = new StringBuilder();
toStringHelper( sb, 0 );
return sb.toString();
}
private void toStringHelper( final StringBuilder sb, final int depth )
{
depth( sb, depth ).append("name=").append( name ).append(", type=").append( type )
.append( ", contents=").append( contents ).append(", offset=").append( offset )
.append(", length=").append( length );
if ( arraySize > 0 )
{
sb.append(", arrayBase=").append( arrayBase );
sb.append(", arrayElemSize=").append( arrayElementSize );
sb.append( ", arraySize=").append( arraySize );
}
for ( final ObjectInfo child : children )
{
sb.append( '\n' );
child.toStringHelper(sb, depth + 1);
}
}
private StringBuilder depth( final StringBuilder sb, final int depth )
{
for ( int i = 0; i < depth; ++i )
sb.append( "\t");
return sb;
}
private long addPaddingSize(long size){
if(size % 8 != 0){
return (size / 8 + 1) * 8;
}
return size;
}
}
package test;
import java.lang.reflect.Array;
import java.lang.reflect.Field;
import java.lang.reflect.Modifier;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
import java.util.IdentityHashMap;
import java.util.List;
import java.util.Map;
import sun.misc.Unsafe;
/**
* This class could be used for any object contents/memory layout printing.
*/
public class ClassIntrospector {
private static final Unsafe unsafe;
/** Size of any Object reference */
private static final int objectRefSize;
static {
try {
Field field = Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (Unsafe) field.get(null);
objectRefSize = unsafe.arrayIndexScale(Object[].class);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/** Sizes of all primitive values */
private static final Map<Class, Integer> primitiveSizes;
static {
primitiveSizes = new HashMap<Class, Integer>(10);
primitiveSizes.put(byte.class, 1);
primitiveSizes.put(char.class, 2);
primitiveSizes.put(int.class, 4);
primitiveSizes.put(long.class, 8);
primitiveSizes.put(float.class, 4);
primitiveSizes.put(double.class, 8);
primitiveSizes.put(boolean.class, 1);
}
/**
* Get object information for any Java object. Do not pass primitives to
* this method because they will boxed and the information you will get will
* be related to a boxed version of your value.
*
* @param obj
* Object to introspect
* @return Object info
* @throws IllegalAccessException
*/
public ObjectInfo introspect(final Object obj)
throws IllegalAccessException {
try {
return introspect(obj, null);
} finally { // clean visited cache before returning in order to make
// this object reusable
m_visited.clear();
}
}
// we need to keep track of already visited objects in order to support
// cycles in the object graphs
private IdentityHashMap<Object, Boolean> m_visited = new IdentityHashMap<Object, Boolean>(
100);
private ObjectInfo introspect(final Object obj, final Field fld)
throws IllegalAccessException {
// use Field type only if the field contains null. In this case we will
// at least know what's expected to be
// stored in this field. Otherwise, if a field has interface type, we
// won't see what's really stored in it.
// Besides, we should be careful about primitives, because they are
// passed as boxed values in this method
// (first arg is object) - for them we should still rely on the field
// type.
boolean isPrimitive = fld != null && fld.getType().isPrimitive();
boolean isRecursive = false; // will be set to true if we have already
// seen this object
if (!isPrimitive) {
if (m_visited.containsKey(obj))
isRecursive = true;
m_visited.put(obj, true);
}
final Class type = (fld == null || (obj != null && !isPrimitive)) ? obj
.getClass() : fld.getType();
int arraySize = 0;
int baseOffset = 0;
int indexScale = 0;
if (type.isArray() && obj != null) {
baseOffset = unsafe.arrayBaseOffset(type);
indexScale = unsafe.arrayIndexScale(type);
arraySize = baseOffset + indexScale * Array.getLength(obj);
}
final ObjectInfo root;
if (fld == null) {
root = new ObjectInfo("", type.getCanonicalName(), getContents(obj,
type), 0, getShallowSize(type), arraySize, baseOffset,
indexScale);
} else {
final int offset = (int) unsafe.objectFieldOffset(fld);
root = new ObjectInfo(fld.getName(), type.getCanonicalName(),
getContents(obj, type), offset, getShallowSize(type),
arraySize, baseOffset, indexScale);
}
if (!isRecursive && obj != null) {
if (isObjectArray(type)) {
// introspect object arrays
final Object[] ar = (Object[]) obj;
for (final Object item : ar)
if (item != null)
root.addChild(introspect(item, null));
} else {
for (final Field field : getAllFields(type)) {
if ((field.getModifiers() & Modifier.STATIC) != 0) {
continue;
}
field.setAccessible(true);
root.addChild(introspect(field.get(obj), field));
}
}
}
root.sort(); // sort by offset
return root;
}
// get all fields for this class, including all superclasses fields
private static List<Field> getAllFields(final Class type) {
if (type.isPrimitive())
return Collections.emptyList();
Class cur = type;
final List<Field> res = new ArrayList<Field>(10);
while (true) {
Collections.addAll(res, cur.getDeclaredFields());
if (cur == Object.class)
break;
cur = cur.getSuperclass();
}
return res;
}
// check if it is an array of objects. I suspect there must be a more
// API-friendly way to make this check.
private static boolean isObjectArray(final Class type) {
if (!type.isArray())
return false;
if (type == byte[].class || type == boolean[].class
|| type == char[].class || type == short[].class
|| type == int[].class || type == long[].class
|| type == float[].class || type == double[].class)
return false;
return true;
}
// advanced toString logic
private static String getContents(final Object val, final Class type) {
if (val == null)
return "null";
if (type.isArray()) {
if (type == byte[].class)
return Arrays.toString((byte[]) val);
else if (type == boolean[].class)
return Arrays.toString((boolean[]) val);
else if (type == char[].class)
return Arrays.toString((char[]) val);
else if (type == short[].class)
return Arrays.toString((short[]) val);
else if (type == int[].class)
return Arrays.toString((int[]) val);
else if (type == long[].class)
return Arrays.toString((long[]) val);
else if (type == float[].class)
return Arrays.toString((float[]) val);
else if (type == double[].class)
return Arrays.toString((double[]) val);
else
return Arrays.toString((Object[]) val);
}
return val.toString();
}
// obtain a shallow size of a field of given class (primitive or object
// reference size)
private static int getShallowSize(final Class type) {
if (type.isPrimitive()) {
final Integer res = primitiveSizes.get(type);
return res != null ? res : 0;
} else
return objectRefSize;
}
} First, a test class is used to verify the results calculated by Unsafe
public class ClassIntrospectorTest
{
public static void main(String[] args) throws IllegalAccessException {
final ClassIntrospector ci = new ClassIntrospector();
ObjectInfo res;
res = ci.introspect( new ObjectA() );
System.out.println( res.getDeepSize() );
}
private static class ObjectA {
String str; // 4
int i1; // 4
byte b1; // 1
byte b2; // 1
int i2; // 4
ObjectB obj; //4
byte b3; // 1
}
private static class ObjectB {
}
} The calculation results are as follows:
32
It is consistent with our previous calculation results and proved to be correct.
Finally, test the length of the array object. There are two classes as follows:
private static class ObjectC {
ObjectD[] array = new ObjectD[2];
}
private static class ObjectD {
int value;
} Their general distribution in memory is shown in the figure below:
We can manually calculate the size of ObjectC obj = new ObjectC():
Small size of ObjectC = 8 (_mark) + 4 (OOP pointer) + 4(ObjectD [] reference) = 16
Length of new ObjectD[2] array = 8(_mark) + 4(oop pointer) + 4 (array length accounts for 4 bytes) + 4(ObjectD[0] reference) + 4(ObjectD[1] reference) = 24
Since the ObjectD [] array does not point to a specific object size, the result of our manual calculation is 16 + 24 = 40
Use the Unsafe object to calculate:
public static void main(String[] args) throws IllegalAccessException {
final ClassIntrospector ci = new ClassIntrospector();
ObjectInfo res;
res = ci.introspect( new ObjectC() );
System.out.println( res.getDeepSize() );
} The calculation results are as follows, which are consistent with our calculation results and proved to be correct:
40
Then point the ObjectD [] array to the specific ObjectD object, and then test the results:
public static void main(String[] args) throws IllegalAccessException {
final ClassIntrospector ci = new ClassIntrospector();
ObjectInfo res;
res = ci.introspect( new ObjectC() );
System.out.println( res.getDeepSize() );
}
private static class ObjectC {
ObjectD[] array = new ObjectD[2];
public ObjectC(){
array[0] = new ObjectD();
array[1] = new ObjectD();
}
}
private static class ObjectD {
int value;
} We can manually calculate the size of ObjectC obj = new ObjectC():
Small size of ObjectC = 8 (_mark) + 4 (OOP pointer) + 4(ObjectD [] reference) = 16
Length of new ObjectD[2] array = 8(_mark) + 4(oop pointer) + 4 (array length accounts for 4 bytes) + 4(ObjectD[0] reference) + 4(ObjectD[1] reference) = 24
ObjectD object length = 8(_mark) + 4(oop pointer) + 4(value) = 16
So the space actually occupied by ObjectC = 16 + 24 + 2 * 16 = 72
The result calculated by Unsafe method is also 72, which is consistent with our manual calculation method.
Reference: Memory introspection using sun.misc.Unsafe and reflection