memory pressureissues materialize are:
- Too many allocations, incorrect data representation
- Heavy usage of reflection or strings
- Using globals
- Orphaned, never-ending goroutines
利用golang实现blockchain
Since in Go there is no simple way to analyze the full core dump, getting to the roots of an object that does not get GC-ed is difficult.
It is more important to build the system in a way that avoids premature optimizations and enables you to perform them later on as the code matures, rather than over engineer it from the get-go.
Go has several built in profiles for us to use in common cases:
- goroutine — stack traces of all current goroutines
- heap — a sampling of memory allocations of live objects
- allocs — a sampling of all past memory allocations
- threadcreate — stack traces that led to the creation of new OS threads
- block — stack traces that led to blocking on synchronization primitives
- mutex — stack traces of holders of contended mutexes
(pprof) top
Showing nodes accounting for 330.04MB, 93.73% of 352.11MB total
Dropped 19 nodes (cum <= 1.76MB)
Showing top 10 nodes out of 56
flat flat% sum% cum cum%
142.02MB 40.33% 40.33% 142.02MB 40.33% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/membuffers/go.(*InternalMessage).lazyCalcOffsets
28MB 7.95% 48.29% 28MB 7.95% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.TransactionsBlockProofReader (inline)
26.51MB 7.53% 55.81% 39.01MB 11.08% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.(*ResultsBlockHeaderBuilder).Build
25.51MB 7.24% 63.06% 32.51MB 9.23% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.(*ResultsBlockProofBuilder).Build
23MB 6.53% 69.59% 23MB 6.53% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.ResultsBlockHeaderReader (inline)
20.50MB 5.82% 75.41% 20.50MB 5.82% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.TransactionsBlockMetadataReader (inline)
20MB 5.68% 81.09% 20MB 5.68% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.TransactionsBlockHeaderReader (inline)
16MB 4.54% 85.64% 24MB 6.82% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.(*TransactionsBlockHeaderBuilder).Build
14.50MB 4.12% 89.76% 122.51MB 34.79% github.com/orbs-network/orbs-network-go/services/gossip/codec.DecodeBlockPairs
14MB 3.98% 93.73% 14MB 3.98% github.com/orbs-network/orbs-network-go/vendor/github.com/orbs-network/orbs-spec/types/go/protocol.ResultsBlockProofReader (inline)- flat means that the memory allocated by this function and is held by that function
- cum means that the memory was allocated by this function or function that it called down the stack
understanding that when we serialize data, meaning that we allocate memory to structs and primitive objects (int, string), it is never released.在数据序列化过城中，相关的memory还依旧被保存着