There will be one or more new tables in PERFORMANCE_SCHEMA showing
what MySQL is doing with memory.
MEMORY table
------------
There is one row for each memory pool, which may mean there
is a row for global, a lot of rows for per-thread, and separate
rows for different pools. All counts are in bytes.
MEMORY_TYPE VARCHAR MySQL or its storage engines use different
allocation methods, and they might change
depending on version or on compilation.
Examples: "TCMalloc", "safemalloc".
MEMORY_POOL VARCHAR A descriptor. Usually NULL. In an ideal world,
we'd know if the memory was specifically for
"buffers for storage engine X", etc.
MEMORY_THREAD INT If it's global (process) memory, this is NULL.
If it's per-thread memory, this is thread
number, which may or may not be relatable to
PROCESSLIST.
MEMORY_MAX INT Maximum. If there is no such thing
as a maximum, this is NULL.
MEMORY_USE INT How much is in use currently.
MEMORY_USE_MAX INT How much was in use at peak. High water mark.
MEMORY_USE_AVG INT How much was in average usse, according to
snapshots.
MEMORY_BLOCK_SIZE INT And whatever else you see in MEM_ROOT.
Getting the information
-----------------------
It depends on how we're allocating.
In include/my_alloc.h there is a structure MEM_ROOT, which has
blocks free, used, and so on. A patch suggestion (BUG#43237)
proposes adding fields from all MEM_ROOT instances into a global
counter and using SHOW STATUS to display.
Sometimes the malloc/free library has a way to look at details,
for example TCMalloc's heap profiler
http://goog-perftools.sourceforge.net/doc/heap_profiler.html
If you just use GNU malloc(), then GNU mallinfo() will tell you
http://www.gnu.org/software/libtool/manual/libc/Statistics-of-Malloc.html
In WL#682 "Memory consumption estimation for x86/32 Linux",
there is a suggestion about getrusage().
Instrumenting malloc/free calls
-------------------------------
Primarily this worklog task is about the state of memory.
We must also consider whether to monitor actions that affect
memory, which for convention we refer to as "malloc" and "free" calls.
WL#2360 mentions "memory malloc/free" as a possible class for
events in performance_schema. The main thing we'd be monitoring
here is elapsed time, but of course the amount would be in too.
In the PERFORMANCE_SCHEMA.EVENTS_WAITS_CURRENT table, we'd have:
EVENT_NAME "Wait/Memory/Allocator/safemalloc(etc.)/Server"
SOURCE as usual
TIMER_START as usual
TIMER_END as usual
TIMER_WAIT as usual
SPINS perhaps the amount that's being allocated
OBJECT_... NULL, except OBJECT_INSTANCE_ID = memory location
NESTING_EVENT_ID as usual
And its aggregations per thread etc. are, as usual, as usual.
One possible objection to this table is that there might be no
"wait" -- no operating system call, no significant time consumption.
We'll have to do some tests before deciding we really want this.
It will be interesting to see what "total" bytes allocated/freed
is according to a summary table of EVENTS_WAITS_CURRENT, but it's
you can't depend on such totals. Some memory might be released
when a thread dies, or when a memory pool is reorganized.
Instrumenting memset/memcpy/strcpy etc.
---------------------------------------
There is a possible analogy with the specification for
WL#4678 "PERFORMANCE_SCHEMA Instrumenting File IO".
For files, we instrument "create file" + "read/write file".
So for memory, we should instrument "create memory" i.e. malloc
+ "read/write memory" i.e. memset, memcpy, strcpy, and other
functions that read/write memory. Naturally we would only want
to instrument a very few such calls, when the implementor believes
that the relative overhead is slow because there are many bytes,
or perhaps in a special version with a compile option that by
default is OFF (implementor-defined).
For the name, a possible example is:
"wait/memory/unmapped-global/maria/page-buffer"
In fact the decision about the part of a name that follows
"maria" would be entirely the business of Maria programmers.
But for the parts of the name that precede "maria", we want
to have a public worklog description like this one, because
we don't want other storage engines to implement essentially
the same thing with different names. By using worklog as a
central registry, we ensure that a query for "all memory
operations" will find memset done by all storage engines together.
List of columns and data types:
THREAD_ID as for mutexes
EVENT_ID as for mutexes
EVENT_NAME see above
SOURCE as for mutexes
TIMER_START as for mutexes
TIMER_END as for mutexes
TIMER_WAIT as for mutexes
SPINS NULL
OBJECT_SCHEMA implementor-defined, probably NULL
OBJECT_NAME implementor-defined, probably NULL
OBJECT_TYPE implementor-defined, probably NULL
OBJECT_INSTANCE_BEGIN memory address (if it's memcpy, use target address)
NESTING_EVENT_ID as for mutexes
Original worklog description
----------------------------
The original description for this worklog task was:
"
Due to a request from Brian Aker ...
SHOW [FULL] PROCESSLIST should have
a new field which indicates the
estimated amount of memory that the
connection is using.
There should be an equivalent column
in INFORMATION_SCHEMA.PROCESSLIST.
Add a new column to SHOW FULL PROCESSLIST
Add a new column to INFORMATION_SCHEMA.PROCESSLIST
- For SHOW, the new column is Memory_Used
- For INFORMATION_SCHEMA, the new column is MEMORY_USED
- Datatype for each column is INT
- The column will display the estimated amount of memory
used by the connection
- The Memory_Used column should be displayed only with
SHOW FULL PROCESSLIST, and not with SHOW PROCESSLIST
"
The current plan is very different. No use of SHOW, no use
of INFORMATION_SCHEMA, no connection to PROCESSLIST.
But there's no hurry to change the worklog task's title.
References
----------
WL#682 "Memory consumption estimation for x86/32 Linux"
WL#3675 "Falcon Diagnostics"
BUG#41096 Agent needs DTrace probes for all dynamically allocated structures
BUG#43237 "Add Bytes_allocated to SHOW STATUS for #bytes currently
allocated in MEM_ROOT" (feature request)
dev-private thread "Memory allocation debugging patch"
[mysql intranet]/secure/mailarchive/mail.php?folder=4&mail=15631
dev-private thread "Re: Memory consumption"
[mysql intranet]/secure/mailarchive/mail.php?folder=4&mail=8695
You must be logged in to tag this worklog