diff options
author | 2014-12-07 21:41:33 +0100 | |
---|---|---|
committer | 2014-12-18 09:54:37 +0100 | |
commit | e37c698270633327245beb0fbd8699db8a4b65b4 (patch) | |
tree | ed83173785e55c3f91467a19d8c4033e723a98e9 | |
parent | kernel: Provide READ_ONCE and ASSIGN_ONCE (diff) | |
download | linux-e37c698270633327245beb0fbd8699db8a4b65b4.tar.gz linux-e37c698270633327245beb0fbd8699db8a4b65b4.tar.bz2 linux-e37c698270633327245beb0fbd8699db8a4b65b4.zip |
mm: replace ACCESS_ONCE with READ_ONCE or barriers
ACCESS_ONCE does not work reliably on non-scalar types. For
example gcc 4.6 and 4.7 might remove the volatile tag for such
accesses during the SRA (scalar replacement of aggregates) step
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58145)
Let's change the code to access the page table elements with
READ_ONCE that does implicit scalar accesses for the gup code.
mm_find_pmd is tricky, because m68k and sparc(32bit) define pmd_t
as array of longs. This code requires just that the pmd_present
and pmd_trans_huge check are done on the same value, so a barrier
is sufficent.
A similar case is in handle_pte_fault. On ppc44x the word size is
32 bit, but a pte is 64 bit. A barrier is ok as well.
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: linux-mm@kvack.org
Acked-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
-rw-r--r-- | mm/gup.c | 2 | ||||
-rw-r--r-- | mm/memory.c | 11 | ||||
-rw-r--r-- | mm/rmap.c | 3 |
3 files changed, 13 insertions, 3 deletions
@@ -917,7 +917,7 @@ static int gup_pud_range(pgd_t *pgdp, unsigned long addr, unsigned long end, pudp = pud_offset(pgdp, addr); do { - pud_t pud = ACCESS_ONCE(*pudp); + pud_t pud = READ_ONCE(*pudp); next = pud_addr_end(addr, end); if (pud_none(pud)) diff --git a/mm/memory.c b/mm/memory.c index 3e503831e042..d86aa88902a0 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3202,7 +3202,16 @@ static int handle_pte_fault(struct mm_struct *mm, pte_t entry; spinlock_t *ptl; - entry = ACCESS_ONCE(*pte); + /* + * some architectures can have larger ptes than wordsize, + * e.g.ppc44x-defconfig has CONFIG_PTE_64BIT=y and CONFIG_32BIT=y, + * so READ_ONCE or ACCESS_ONCE cannot guarantee atomic accesses. + * The code below just needs a consistent view for the ifs and + * we later double check anyway with the ptl lock held. So here + * a barrier will do. + */ + entry = *pte; + barrier(); if (!pte_present(entry)) { if (pte_none(entry)) { if (vma->vm_ops) { diff --git a/mm/rmap.c b/mm/rmap.c index 19886fb2f13a..1e542744e131 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -581,7 +581,8 @@ pmd_t *mm_find_pmd(struct mm_struct *mm, unsigned long address) * without holding anon_vma lock for write. So when looking for a * genuine pmde (in which to find pte), test present and !THP together. */ - pmde = ACCESS_ONCE(*pmd); + pmde = *pmd; + barrier(); if (!pmd_present(pmde) || pmd_trans_huge(pmde)) pmd = NULL; out: |