KAMEZAWA Hiroyuki 569b846df5
memcg: coalesce uncharge during unmap/truncate ...
In massive parallel enviroment, res_counter can be a performance
bottleneck. One strong techinque to reduce lock contention is reducing
calls by coalescing some amount of calls into one.
Considering charge/uncharge chatacteristic,
- charge is done one by one via demand-paging.
- uncharge is done by
- in chunk at munmap, truncate, exit, execve...
- one by one via vmscan/paging.
It seems we have a chance to coalesce uncharges for improving scalability
at unmap/truncation.
This patch is a for coalescing uncharge. For avoiding scattering memcg's
structure to functions under /mm, this patch adds memcg batch uncharge
information to the task. A reason for per-task batching is for making use
of caller's context information. We do batched uncharge (deleyed
uncharge) when truncation/unmap occurs but do direct uncharge when
uncharge is called by memory reclaim (vmscan.c).
The degree of coalescing depends on callers
- at invalidate/trucate... pagevec size
- at unmap ....ZAP_BLOCK_SIZE
(memory itself will be freed in this degree.)
Then, we'll not coalescing too much.
On x86-64 8cpu server, I tested overheads of memcg at page fault by
running a program which does map/fault/unmap in a loop. Running
a task per a cpu by taskset and see sum of the number of page faults
in 60secs.
[without memcg config]
40156968 page-faults # 0.085 M/sec ( +- 0.046% )
27.67 cache-miss/faults
[root cgroup]
36659599 page-faults # 0.077 M/sec ( +- 0.247% )
31.58 miss/faults
[in a child cgroup]
18444157 page-faults # 0.039 M/sec ( +- 0.133% )
69.96 miss/faults
[child with this patch]
27133719 page-faults # 0.057 M/sec ( +- 0.155% )
47.16 miss/faults
We can see some amounts of improvement.
(root cgroup doesn't affected by this patch)
Another patch for "charge" will follow this and above will be improved more.
Changelog(since 2009/10/02):
- renamed filed of memcg_batch (as pages to bytes, memsw to memsw_bytes)
- some clean up and commentary/description updates.
- added initialize code to copy_process(). (possible bug fix)
Changelog(old):
- fixed !CONFIG_MEM_CGROUP case.
- rebased onto the latest mmotm + softlimit fix patches.
- unified patch for callers
- added commetns.
- make ->do_batch as bool.
- removed css_get() at el. We don't need it.
Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com >
Cc: Balbir Singh <balbir@in.ibm.com >
Cc: Daisuke Nishimura <nishimura@mxp.nes.nec.co.jp >
Signed-off-by: Andrew Morton <akpm@linux-foundation.org >
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org >
2009-12-16 07:20:07 -08:00
2009-09-22 20:49:04 +01:00
2009-11-08 01:50:21 -08:00
2009-09-19 00:51:34 -03:00
2009-12-14 00:26:26 +01:00
2009-12-09 19:52:01 -08:00
2009-09-22 07:54:33 -07:00
2009-12-15 08:59:33 -08:00
2009-11-18 22:19:03 -08:00
2009-11-24 08:18:54 +02:00
2009-12-03 13:23:12 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-09-23 07:39:29 -07:00
2009-12-14 12:49:47 +11:00
2009-09-22 13:32:45 +01:00
2009-12-14 10:22:11 -08:00
2009-10-27 16:47:55 -04:00
2009-12-03 15:58:56 -05:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-09-24 07:21:05 -07:00
2009-12-11 15:34:40 -08:00
2009-10-19 15:55:59 +09:00
2009-09-19 02:14:45 -04:00
2009-09-27 11:39:25 -07:00
2009-09-20 16:09:20 +05:30
2009-09-23 07:39:29 -07:00
2009-12-03 18:01:23 -05:00
2009-09-30 16:12:20 -07:00
2009-12-15 08:53:35 -08:00
2009-10-29 13:59:26 +01:00
2009-09-21 21:03:58 +01:00
2009-09-24 07:21:01 -07:00
2009-11-26 09:16:19 +01:00
2009-12-03 09:24:48 +01:00
2009-10-01 21:19:34 +02:00
2009-11-10 12:31:43 +01:00
2009-11-02 23:39:15 -08:00
2009-11-24 15:06:47 +11:00
2009-12-04 23:41:47 +01:00
2009-12-03 19:28:51 +01:00
2009-10-01 16:11:11 -07:00
2009-12-04 15:39:55 +01:00
2009-11-13 20:46:24 +01:00
2009-12-08 19:28:09 -08:00
2009-09-23 07:39:29 -07:00
2009-11-08 20:57:03 -08:00
2009-12-05 15:32:03 -08:00
2009-11-02 15:47:54 +01:00
2009-12-05 15:32:03 -08:00
2009-09-24 07:20:57 -07:00
2009-11-25 17:57:36 +01:00
2009-12-09 17:09:36 +11:00
2009-11-24 13:33:34 -05:00
2009-12-06 21:10:56 +01:00
2009-09-23 11:01:25 -07:00
2009-10-19 12:58:55 +09:00
2009-12-02 22:07:23 -08:00
2009-12-15 08:53:28 -08:00
2009-12-15 08:53:32 -08:00
2009-09-19 13:13:31 -07:00
2009-09-24 07:20:57 -07:00
2009-09-18 09:48:52 -07:00
2009-12-10 23:52:27 +00:00
2009-12-11 11:24:51 -08:00
2009-12-10 23:52:01 +00:00
2009-12-10 23:52:22 +00:00
2009-11-09 09:40:57 +01:00
2009-12-10 23:52:05 +00:00
2009-12-08 19:26:55 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 15:20:57 +01:00
2009-10-01 21:17:49 +02:00
2009-10-01 21:17:49 +02:00
2009-11-25 17:57:36 +01:00
2009-12-15 08:53:25 -08:00
2009-12-15 08:53:33 -08:00
2009-10-06 10:35:10 +02:00
2009-12-15 08:53:27 -08:00
2009-11-04 09:50:58 -08:00
2009-12-03 13:18:04 -08:00
2009-09-23 07:39:29 -07:00
2009-12-10 15:02:51 +01:00
2009-11-11 15:22:49 +01:00
2009-12-10 15:02:53 +01:00
2009-12-10 09:34:40 -08:00
2009-12-03 12:14:36 -08:00
2009-11-04 09:50:58 -08:00
2009-10-31 11:40:51 +01:00
2009-12-11 11:24:52 -08:00
2009-09-22 07:17:47 -07:00
2009-12-10 15:02:50 +01:00
2009-11-19 18:12:05 +00:00
2009-11-19 18:11:35 +00:00
2009-11-24 17:00:27 +11:00
2009-12-09 13:55:26 -05:00
2009-10-07 15:52:09 -04:00
2009-10-08 12:16:35 -07:00
2009-11-04 09:50:58 -08:00
2009-11-10 11:50:21 +01:00
2009-09-23 07:39:29 -07:00
2009-12-03 11:58:47 +00:00
2009-12-08 20:30:41 -08:00
2009-12-16 07:20:01 -08:00
2009-10-26 09:40:29 +01:00
2009-11-05 14:08:03 +01:00
2009-12-14 23:55:34 +01:00
2009-12-15 08:53:12 -08:00
2009-12-14 10:10:55 -08:00
2009-12-01 14:47:32 +08:00
2009-09-19 00:16:14 -03:00
2009-11-20 00:25:42 +00:00
2009-12-14 21:17:29 +01:00
2009-09-17 23:23:45 -07:00
2009-11-17 10:16:32 -05:00
2009-12-02 21:58:33 -08:00
2009-11-28 15:05:05 -05:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-12-08 20:42:28 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-26 15:53:10 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-11-26 16:00:36 -08:00
2009-11-19 11:08:54 -05:00
2009-11-04 09:50:58 -08:00
2009-12-09 19:43:33 -08:00
2009-11-04 09:50:58 -08:00
2009-12-03 12:14:38 -08:00
2009-12-15 09:02:01 -08:00
2009-10-02 15:42:19 +02:00
2009-11-20 00:52:09 -08:00
2009-12-02 23:38:13 -08:00
2009-11-04 14:22:21 +01:00
2009-12-04 16:36:18 +01:00
2009-09-23 07:39:41 -07:00
2009-11-04 09:50:58 -08:00
2009-09-26 10:17:19 -07:00
2009-10-18 18:52:53 -07:00
2009-12-14 23:55:33 +01:00
2009-10-23 08:28:10 +02:00
2009-11-15 22:23:47 -08:00
2009-12-11 15:18:07 -08:00
2009-09-30 00:32:06 -04:00
2009-09-23 07:39:58 -07:00
2009-11-26 12:59:20 +01:00
2009-12-15 08:53:26 -08:00
2009-12-11 15:18:03 -08:00
2009-10-25 17:31:30 +01:00
2009-12-15 08:53:26 -08:00
2009-09-19 13:13:17 -07:00
2009-09-23 07:39:29 -07:00
2009-09-23 23:08:43 +02:00
2009-09-26 10:17:19 -07:00
2009-12-15 08:53:20 -08:00
2009-12-03 09:32:17 +02:00
2009-12-09 17:14:38 +11:00
2009-12-09 19:42:25 -08:00
2009-09-17 23:23:45 -07:00
2009-09-21 06:27:08 +02:00
2009-12-15 08:53:36 -08:00
2009-10-01 21:17:49 +02:00
2009-11-10 09:33:46 +11:00
2009-09-23 07:39:42 -07:00
2009-12-16 07:20:07 -08:00
2009-12-15 08:53:20 -08:00
2009-12-15 08:53:12 -08:00
2009-09-22 07:17:35 -07:00
2009-12-15 08:53:20 -08:00
2009-09-19 12:50:38 -07:00
2009-09-21 15:14:56 +02:00
2009-09-22 07:17:37 -07:00
2009-09-27 11:39:25 -07:00
2009-12-15 08:53:24 -08:00
2009-09-22 07:17:42 -07:00
2009-09-22 07:17:31 -07:00
2009-09-24 07:21:04 -07:00
2009-09-23 07:39:43 -07:00
2009-09-25 00:32:58 +09:30
2009-10-29 08:56:17 +10:30
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-12-11 11:24:54 -08:00
2009-11-04 09:50:58 -08:00
2009-12-05 15:22:26 -08:00
2009-12-03 15:59:22 -08:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-12-05 16:08:40 -05:00
2009-12-04 15:55:38 -05:00
2009-09-26 10:17:19 -07:00
2009-12-05 16:08:40 -05:00
2009-11-20 10:05:52 +09:00
2009-11-28 15:05:05 -05:00
2009-11-06 14:32:24 +03:00
2009-12-15 08:53:13 -08:00
2009-12-15 08:53:13 -08:00
2009-12-01 16:15:50 -08:00
2009-12-15 08:53:12 -08:00
2009-10-15 10:58:07 -06:00
2009-10-15 10:58:09 -06:00
2009-12-09 12:12:44 +02:00
2009-12-16 07:19:57 -08:00
2009-09-24 07:20:58 -07:00
2009-12-15 08:53:17 -08:00
2009-12-12 14:27:24 -08:00
2009-11-04 13:06:36 -08:00
2009-12-14 10:11:34 -08:00
2009-12-04 15:56:19 -08:00
2009-10-29 22:34:12 +09:00
2009-12-08 10:02:12 +09:00
2009-11-15 09:51:24 +01:00
2009-12-14 23:55:34 +01:00
2009-09-24 15:38:57 -07:00
2009-11-04 09:50:58 -08:00
2009-11-04 09:50:58 -08:00
2009-12-04 15:39:55 +01:00
2009-12-11 11:24:55 -08:00
2009-12-14 23:55:33 +01:00
2009-12-06 16:17:57 +01:00
2009-12-15 08:53:25 -08:00
2009-09-22 07:17:49 -07:00
2009-09-22 07:17:47 -07:00
2009-10-04 15:05:10 -07:00
2009-12-03 11:43:23 +00:00
2009-10-04 03:23:17 +02:00
2009-12-02 09:55:33 +01:00
2009-09-23 07:39:41 -07:00
2009-12-10 15:02:54 +01:00
2009-09-22 07:17:24 -07:00
2009-09-27 11:39:25 -07:00
2009-10-23 17:26:37 +02:00
2009-09-19 08:53:22 +02:00
2009-11-10 22:26:29 -08:00
2009-11-22 18:58:16 +01:00
2009-11-22 18:58:16 +01:00
2009-11-22 18:58:16 +01:00
2009-10-30 15:06:37 -07:00
2009-12-16 07:20:06 -08:00
2009-09-24 07:20:57 -07:00
2009-10-01 16:11:13 -07:00
2009-11-18 17:09:26 -05:00
2009-12-15 08:53:20 -08:00
2009-11-04 09:50:58 -08:00
2009-12-14 23:55:33 +01:00
2009-11-04 09:50:58 -08:00
2009-12-14 23:55:33 +01:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:33 +01:00
2009-12-15 08:53:26 -08:00
2009-12-16 07:20:07 -08:00
2009-11-13 14:23:11 -08:00
2009-11-23 15:53:52 -05:00
2009-10-30 08:27:25 +11:00
2009-11-10 09:33:46 +11:00
2009-09-24 07:47:35 -04:00
2009-09-19 13:13:19 -07:00
2009-11-04 08:44:54 -08:00
2009-12-04 15:39:55 +01:00
2009-09-19 13:13:30 -07:00
2009-10-26 11:01:30 -07:00
2009-11-02 15:43:20 +09:00
2009-11-30 12:02:53 +09:00
2009-09-15 09:50:49 -07:00
2009-09-24 07:21:01 -07:00
2009-12-02 19:57:15 -08:00
2009-12-11 09:17:02 +01:00
2009-12-01 08:20:31 -08:00
2009-12-11 09:17:02 +01:00
2009-12-16 07:20:04 -08:00
2009-09-28 18:00:48 +02:00
2009-11-18 14:52:25 +01:00
2009-10-13 11:48:32 -07:00
2009-10-29 03:00:06 -07:00
2009-12-14 23:55:33 +01:00
2009-12-14 23:55:33 +01:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:33 +01:00
2009-10-26 09:40:30 +01:00
2009-12-15 08:53:34 -08:00
2009-11-17 17:40:33 -08:00
2009-12-15 08:53:16 -08:00
2009-09-16 11:50:05 +02:00
2009-11-15 09:03:10 +01:00
2009-12-11 06:44:29 -05:00
2009-12-09 19:43:33 -08:00
2009-09-18 09:48:52 -07:00
2009-12-02 22:07:25 -08:00
2009-11-13 20:46:24 +01:00
2009-12-16 07:20:00 -08:00
2009-11-13 20:46:24 +01:00
2009-12-08 19:30:19 -08:00
2009-10-14 15:02:34 +02:00
2009-10-29 11:17:40 +11:00
2009-12-09 14:05:49 -05:00
2009-09-24 07:21:00 -07:00
2009-11-26 09:04:55 +01:00
2009-09-27 13:35:16 -07:00
2009-12-15 08:53:28 -08:00
2009-11-20 00:52:05 -08:00
2009-11-10 20:54:38 -08:00
2009-12-11 11:55:26 -08:00
2009-12-11 11:55:27 -08:00
2009-12-10 22:55:37 +01:00
2009-11-29 22:03:04 +01:00
2009-09-23 18:13:10 -07:00
2009-09-24 09:57:08 -07:00
2009-12-09 19:50:49 -08:00
2009-10-22 16:39:28 +10:30
2009-10-22 16:39:28 +10:30
2009-10-22 16:39:30 +10:30
2009-09-23 07:39:29 -07:00
2009-10-22 16:39:28 +10:30
2009-09-23 22:26:32 +09:30
2009-10-22 16:39:28 +10:30
2009-10-22 16:39:28 +10:30
2009-09-23 22:26:31 +09:30
2009-12-15 08:53:16 -08:00
2009-11-13 15:54:27 -08:00
2009-12-15 08:53:28 -08:00
2009-09-23 03:49:27 +04:00
2009-11-16 01:09:48 +09:00
2009-12-03 13:54:25 +01:00
2009-11-18 23:30:41 -08:00
2009-11-25 15:48:37 -08:00
569b846df5