moonjuice/kernel_arpi
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

bpf: Use this_cpu_{inc|dec|inc_return} for bpf_task_storage_busy

Browse Source 
[ Upstream commit 197827a05e13808c60f52632e9887eede63f1c16 ]

Now migrate_disable() does not disable preemption and under some
architectures (e.g. arm64) __this_cpu_{inc|dec|inc_return} are neither
preemption-safe nor IRQ-safe, so for fully preemptible kernel concurrent
lookups or updates on the same task local storage and on the same CPU
may make bpf_task_storage_busy be imbalanced, and
bpf_task_storage_trylock() on the specific cpu will always fail.

Fixing it by using this_cpu_{inc|dec|inc_return} when manipulating
bpf_task_storage_busy.

Fixes: bc235cdb42 ("bpf: Prevent deadlock from recursive bpf_task_storage_[get|delete]")
Signed-off-by: Hou Tao <houtao1@huawei.com>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/r/20220901061938.3789460-2-houtao@huaweicloud.com
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>
This commit is contained in:
Hou Tao
2022-09-01 14:19:35 +08:00
committed by Greg Kroah-Hartman
parent 0e13425104
commit f91e25cfa5
2 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
kernel/bpf/bpf_local_storage.c
View File

@@ -506,11 +506,11 @@ void bpf_local_storage_map_free(struct bpf_local_storage_map *smap,
struct bpf_local_storage_elem, map_node))) { struct bpf_local_storage_elem, map_node))) {
if (busy_counter) { if (busy_counter) {
migrate_disable(); migrate_disable();
__this_cpu_inc(*busy_counter); this_cpu_inc(*busy_counter);
} }
bpf_selem_unlink(selem); bpf_selem_unlink(selem);
if (busy_counter) { if (busy_counter) {
__this_cpu_dec(*busy_counter); this_cpu_dec(*busy_counter);
migrate_enable(); migrate_enable();
} }
cond_resched_rcu(); cond_resched_rcu();

8
kernel/bpf/bpf_task_storage.c
View File

@@ -25,20 +25,20 @@ static DEFINE_PER_CPU(int, bpf_task_storage_busy);
static void bpf_task_storage_lock(void) static void bpf_task_storage_lock(void)
{ {
migrate_disable(); migrate_disable();
__this_cpu_inc(bpf_task_storage_busy); this_cpu_inc(bpf_task_storage_busy);
} }
static void bpf_task_storage_unlock(void) static void bpf_task_storage_unlock(void)
{ {
__this_cpu_dec(bpf_task_storage_busy); this_cpu_dec(bpf_task_storage_busy);
migrate_enable(); migrate_enable();
} }
static bool bpf_task_storage_trylock(void) static bool bpf_task_storage_trylock(void)
{ {
migrate_disable(); migrate_disable();
if (unlikely(__this_cpu_inc_return(bpf_task_storage_busy) != 1)) { if (unlikely(this_cpu_inc_return(bpf_task_storage_busy) != 1)) {
__this_cpu_dec(bpf_task_storage_busy); this_cpu_dec(bpf_task_storage_busy);
migrate_enable(); migrate_enable();
return false; return false;
} }