moonjuice/kernel_arpi
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ANDROID: mm: introduce vma refcounting to protect vma during SPF

Browse Source 
Current mechanism to stabilize a vma during speculative page fault
handling makes a copy of the faulting vma under RCU protection. This
makes it hard to protect elements which do not belong to the vma but
are used by the page fault handler like vma->vm_file.
The problems is that a copy of the vma can't be used to safely
protect the file attached to the original vma unless the file is
also released after RCU grace period (which is how SPF was designed
originally but that caused performance regression and had to be
changed).
To avoid these complications, introduce vma refcounting to stabilize
and operate on the original vma during page fault handling. Page
fault handler finds the vma and increases its refcount under RCU
protection, vma is freed after RCU grace period, vma->vm_file is
released only after refcount indicates no users. This mechanism
guarantees that once get_vma returns a vma, both the vma itself and
vma->vm_file are stable.
Additional benefits of this patch are: we don't need to copy the vma
and no additional logic is needed to stabilize vma->vm_file.

Bug: 257443051
Change-Id: I59d373926d687fcbd56847a8c3500c43bf1844c8
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
This commit is contained in:
Suren Baghdasaryan
2022-11-12 14:34:16 -08:00
parent 50567620db
commit 3f311327f9
7 changed files with 88 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
include/linux/mm.h
View File

@@ -253,6 +253,7 @@ void setup_initial_init_mm(void *start_code, void *end_code,
struct vm_area_struct *vm_area_alloc(struct mm_struct *);
struct vm_area_struct *vm_area_dup(struct vm_area_struct *);
void vm_area_free_no_check(struct vm_area_struct *);
void vm_area_free(struct vm_area_struct *);
#ifndef CONFIG_MMU
@@ -685,6 +686,10 @@ static inline void vma_init(struct vm_area_struct *vma, struct mm_struct *mm)
memset(vma, 0, sizeof(*vma));
vma->vm_mm = mm;
vma->vm_ops = &dummy_vm_ops;
#ifdef CONFIG_SPECULATIVE_PAGE_FAULT
/* Start from 0 to use atomic_inc_unless_negative() in get_vma() */
atomic_set(&vma->file_ref_count, 0);
#endif
INIT_LIST_HEAD(&vma->anon_vma_chain);
}
@@ -3383,6 +3388,9 @@ static inline bool pte_spinlock(struct vm_fault *vmf)
return __pte_map_lock(vmf);
}
struct vm_area_struct *get_vma(struct mm_struct *mm, unsigned long addr);
void put_vma(struct vm_area_struct *vma);
#else /* !CONFIG_SPECULATIVE_PAGE_FAULT */
#define pte_map_lock(___vmf) \