Enabling CONFIG_MODULE_ALLOW_BTF_MISMATCH is an indication that BTF
mismatches are expected and module loading should proceed
anyway. Logging with pr_warn() on every one of these "benign"
mismatches creates unnecessary noise when many such modules are
loaded. Instead, handle this case with a single log warning that BTF
info may be unavailable.
Mismatches also result in calls to __btf_verifier_log() via
__btf_verifier_log_type() or btf_verifier_log_member(), adding several
additional lines of logging per mismatched module. Add checks to these
paths to skip logging for module BTF mismatches in the "allow
mismatch" case.
All existing logging behavior is preserved in the default
CONFIG_MODULE_ALLOW_BTF_MISMATCH=n case.
Signed-off-by: Connor O'Brien <connoro@google.com>
Acked-by: Yonghong Song <yhs@fb.com>
Link: https://lore.kernel.org/r/20230107025331.3240536-1-connoro@google.com
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Bug: 260025057
Bug: 256578296
Bug: 258989318
(cherry picked from commit 9cb61e50bf6bf54db712bba6cf20badca4383f96
https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git
master)
Signed-off-by: Connor O'Brien <connoro@google.com>
Change-Id: I2f9d19a2027610af2e825fc38fd08f56f4cd7091
Modules will need to convert physical addresses in EL2 linear map
addresses, so expose hyp_va to modules.
Bug: 264070847
Change-Id: I641ab3943a029072c351f5a422e2d9d2cbb240be
Signed-off-by: Quentin Perret <qperret@google.com>
Expose an API allowing to query the host stage-2 page-table to module.
Bug: 264070847
Change-Id: Ifbb2c9c8f8899778b9bf1f7df1dae81acba2af6d
Signed-off-by: Quentin Perret <qperret@google.com>
__pkvm_map_module_page() currently goes through a full host-to-hyp
donation for every module page being mapped. However, in order to allow
mapping protected pages as modules, let's make this behaviour optional.
Bug: 264070847
Change-Id: Ic3b1b68f11283c215bc8c6e1352f91bcf52a3935
Signed-off-by: Quentin Perret <qperret@google.com>
Much of the functions related to pKVM modules at EL2 are currently
guarded by the pkvm_modules_lock. This lock is only useful in case of
module hypercalls racing with __pkvm_close_module_registration().
However, the latter is by definition a privileged operation, so we can
trust that the host is not trying to exploit races to attack EL2 at this
point.
As such, let's remove the pkvm_modules_lock and re-use the existing
concept of privileged hypercalls instead. To do so, the hypercall that
limits the privileged range is moved dynamically depending on whether
pKVM modules are supported or not.
Bug: 264070847
Change-Id: I6924471339f2123ab244cdb71ffcb2a299fa75a4
Signed-off-by: Quentin Perret <qperret@google.com>
Now that the pKVM memory-management module APIs are exposed outside of
core pKVM, make sure to sanity check the input to
__pkvm_map_module_page() which expects a VA that was previously
allocated in the private range.
We can't easily check that each private VA page is used to map the
module it was allocated for, but we can at least check that all module
pages fall in the private range, which is still desirable to enforce
some degree of consitency. This is all best effort really.
Bug: 264070847
Change-Id: Ib95e464f57beaf0c32d15075938ca283fe52f720
Signed-off-by: Quentin Perret <qperret@google.com>
Some partners may need to load plug-ins in their modules, which will
require mapping the plugin code in a way that is very similar to how
pKVM maps module code. To ease this use-case, expose the pKVM module
memory-management APIs in the module_ops struct.
Bug: 264070847
Change-Id: I8b30ca50692c6860e3988b847b63ba1553c729cb
Signed-off-by: Quentin Perret <qperret@google.com>
These are dangerous because they assume mask macros are pre-shifted,
GENMASK style. Most of the masks in this kernel tree are not shifted.
Avoid accidental buggy backports in future by turning the SYS_FIELD
accessors into BUILD_BUG().
Signed-off-by: Keir Fraser <keirf@google.com>
Bug: 233588291
Fixes: 1bf4f91abe ("UPSTREAM: arm64/sysreg: Introduce helpers for access to sysreg fields")
Change-Id: I647a66d59279e69c916f66b42705fc20cd256fa8
A partial backport of sysreg naming changes broke extraction of the
L1Ip field from CTR_EL0. Revert the CTR_L1IP() macro to an earlier,
correct form.
Signed-off-by: Keir Fraser <keirf@google.com>
Bug: 233588291
Fixes: f660cd9614 ("BACKPORT: arm64/sysreg: Standardise naming for CTR_EL0 fields")
Change-Id: I8d46ad5e266c5792239066b16fbc78762706c023
Pages on CMA area could have MIGRATE_ISOLATE as well as MIGRATE_CMA
so current is_pinnable_page could miss CMA pages which has MIGRATE_
ISOLATE. It ends up pinning CMA pages as longterm at pin_user_pages
APIs so CMA allocation keep failed until the pin is released.
CPU 0 CPU 1 - Task B
cma_alloc
alloc_contig_range
pin_user_pages_fast(FOLL_LONGTERM)
change pageblock as MIGRATE_ISOLATE
internal_get_user_pages_fast
lockless_pages_from_mm
gup_pte_range
try_grab_folio
is_pinnable_page
return true;
So, pinned the page successfully.
page migration failure with pinned page
..
.. After 30 sec
unpin_user_page(page)
CMA allocation succeeded after 30 sec.
The CMA allocation path protects the migration type change race
using zone->lock but what GUP path need to know is just whether the
page is on CMA area or not rather than exact migration type.
Thus, we don't need zone->lock but just checks migration type in
either of (MIGRATE_ISOLATE and MIGRATE_CMA).
Adding the MIGRATE_ISOLATE check in is_pinnable_page could cause
rejecting of pinning pages on MIGRATE_ISOLATE pageblocks even
though it's neither CMA nor movable zone if the page is temporarily
unmovable. However, such a migration failure by unexpected temporal
refcount holding is general issue, not only come from MIGRATE_ISOLATE
and the MIGRATE_ISOLATE is also transient state like other temporal
elevated refcount problem.
Cc: "Paul E . McKenney" <paulmck@kernel.org>
Cc: David Hildenbrand <david@redhat.com>
Reviewed-by: John Hubbard <jhubbard@nvidia.com>
Signed-off-by: Minchan Kim <minchan@kernel.org>
Conflicts:
include/linux/mm.h
1. There is no is_pinnable_page in 5.10
Link: https://lore.kernel.org/all/20220524171525.976723-1-minchan@kernel.org/
Bug: 231227007
Signed-off-by: Minchan Kim <minchan@google.com>
Change-Id: I5cdd2b8eefdd7e89658abd21c32aa84876ad7782
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit e9dd78ebe1c8e9fcc4067e0795326495a16a9c9b)
Add kasan.page_alloc.sample=10 to CONFIG_CMDLINE in gki_defconfig to make
Hardware Tag-Based (MTE) KASAN tag only one out of every 10 page_alloc
allocations with the order equal or larger than 3, which the omitted
default value for the kasan.page_alloc.sample.order parameter.
As Hardware Tag-Based KASAN is intended to be used in production, its
performance impact is crucial. As page_alloc allocations tend to be big,
tagging and checking all such allocations can introduce a significant
slowdown.
When running a local loopback test on a testing MTE-enabled device in sync
mode, enabling Hardware Tag-Based KASAN introduces a ~50% slowdown.
Setting kasan.page_alloc.sampling to a value higher than 1 allows to lower
the slowdown. The performance improvement saturates around the sampling
interval value of 10 with the default sampling page order of 3, see
b/238286329. This lowers the slowdown to ~20%. The slowdown in real
scenarios involving the network will likely be better.
Enabling page_alloc sampling has a downside: KASAN misses bad accesses to
a page_alloc allocation that has not been tagged. This lowers the value
of KASAN as a security mitigation.
However, based on measuring the number of page_alloc allocations of
different orders during boot in a test build, sampling with the default
kasan.page_alloc.sample.order value affects only ~7% of allocations. The
rest ~93% of allocations are still checked deterministically.
Bug: 238286329
Bug: 264310057
Change-Id: Idfc8600d1f7cc7af28482ff2c8e8ad5ad5948058
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
[The patch is in mm-unstable tree.]
As Hardware Tag-Based KASAN is intended to be used in production, its
performance impact is crucial. As page_alloc allocations tend to be big,
tagging and checking all such allocations can introduce a significant
slowdown.
Add two new boot parameters that allow to alleviate that slowdown:
- kasan.page_alloc.sample, which makes Hardware Tag-Based KASAN tag only
every Nth page_alloc allocation with the order configured by the second
added parameter (default: tag every such allocation).
- kasan.page_alloc.sample.order, which makes sampling enabled by the first
parameter only affect page_alloc allocations with the order equal or
greater than the specified value (default: 3, see below).
The exact performance improvement caused by using the new parameters
depends on their values and the applied workload.
The chosen default value for kasan.page_alloc.sample.order is 3, which
matches both PAGE_ALLOC_COSTLY_ORDER and SKB_FRAG_PAGE_ORDER. This is
done for two reasons:
1. PAGE_ALLOC_COSTLY_ORDER is "the order at which allocations are deemed
costly to service", which corresponds to the idea that only large and
thus costly allocations are supposed to sampled.
2. One of the workloads targeted by this patch is a benchmark that sends
a large amount of data over a local loopback connection. Most multi-page
data allocations in the networking subsystem have the order of
SKB_FRAG_PAGE_ORDER (or PAGE_ALLOC_COSTLY_ORDER).
When running a local loopback test on a testing MTE-enabled device in sync
mode, enabling Hardware Tag-Based KASAN introduces a ~50% slowdown.
Applying this patch and setting kasan.page_alloc.sampling to a value
higher than 1 allows to lower the slowdown. The performance improvement
saturates around the sampling interval value of 10 with the default
sampling page order of 3. This lowers the slowdown to ~20%. The slowdown
in real scenarios involving the network will likely be better.
Enabling page_alloc sampling has a downside: KASAN misses bad accesses to
a page_alloc allocation that has not been tagged. This lowers the value
of KASAN as a security mitigation.
However, based on measuring the number of page_alloc allocations of
different orders during boot in a test build, sampling with the default
kasan.page_alloc.sample.order value affects only ~7% of allocations. The
rest ~93% of allocations are still checked deterministically.
Link: https://lkml.kernel.org/r/129da0614123bb85ed4dd61ae30842b2dd7c903f.1671471846.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Brand <markbrand@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Bug: 238286329
Bug: 264310057
Link: https://lore.kernel.org/all/129da0614123bb85ed4dd61ae30842b2dd7c903f.1671471846.git.andreyknvl@google.com
Change-Id: Icc7befe61848021c68a12034f426f1c300181ad6
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
In the updated HW-wrapped key code in the android14 kernels, HW-wrapped
keys are only allowed on a multi-block-device filesystem if they have a
compatible HW-wrapped keys implementation. While in principle this is a
good thing to check, my implementation of it, which simply checks
whether the block devices have the same crypto profiles, doesn't work
when device-mapper is being used.
To actually do that check correctly, I think we'd need to add a
HW-wrapped keys implementation name or ID to the crypto capabilities.
That being said, in Android the HW-wrapped keys implementation is a
global thing anyway. So in the interest of not overcomplicating things,
for now let's just drop these extra checks that are causing problems.
Bug: 160883801
Bug: 265180564
Fixes: 4887dd4fe3 ("ANDROID: fscrypt: add support for hardware-wrapped keys")
Fixes: 3918b39c3e ("ANDROID: update "block: add basic hardware-wrapped key support" to v7")
Change-Id: Ia49d62cc2c56447fb898f19bf67df1a38af379f8
Signed-off-by: Eric Biggers <ebiggers@google.com>
Add vendor hook for bh_lru and lru_cache_disable
Bug: 238728493
Signed-off-by: Minchan Kim <minchan@google.com>
Change-Id: I81bfad317cf6e8633186ebb3238644306d7a102d
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 74e2ea264cd1895c493b9008b62bfea98dacf3f6)
The pagevec batching causes lru_add_drain_all which is too expensive
sometimes. This patch adds a new vendor hook to drain the pagevec
immediately depending on the page's type.
Bug: 251881967
Signed-off-by: Minchan Kim <minchan@google.com>
Change-Id: Id17e14e69197993ddad511a40c96e51674c02834
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 2f8253b7e6e563cc19cffa120c72f6f528664103)
Add vendor hook for compaction begin/end. The first use would be
to measure compaction durations.
Bug: 229927848
Test: local kernel build test
Signed-off-by: Robin Hsu <robinhsu@google.com>
Change-Id: I3d95434bf49b37199056dc9ddfc36a59a7de17b7
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 13b6bd38bb1f43bfffdb08c8f3a4a20d36ccd670)
CMA first allocation policy for movable makes CMA(Upstream doesn't)
area always full. It's good for memory efficiency since it could use
up CMA available memory most of time. However, it could cause
cma_alloc slow since it causes a lot page migration all the time.
Let's add vendor hook for someone who want to restore CMA allocation
policy to upstream so they will see less page migration in cma_alloc.
If the vendor_hook returns false, the rmqueue_bulk return 0 without
filling pcp->lists so get_populated_pcp_list will return NULL.
Once get_populated_pcp_list returns NULL, __rmqueue_pcplist will retry
the page allocation with original migratetype(currently, original
migratetype couldn't be MIGRATE_CMA) so the retrial will find
available pages from !MIGRATE_CMA free list.
Bug: 231978523
Signed-off-by: Minchan Kim <minchan@google.com>
Change-Id: Ia031d9bc6f34085b892a8d9923bf5b9b1794f94a
Signed-off-by: Richard Chang <richardycc@google.com>
(cherry picked from commit 0ca85e35bf5b4a7ff08f00a060c83e4a82380b64)
Instead of hardcoding queue depth allow user to set the hw queue depth
using module parameter. Set default value to 128 to retain the existing
behavior.
Signed-off-by: Chaitanya Kulkarni <kch@nvidia.com>
Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com>
Link: https://lore.kernel.org/r/20220215213310.7264-5-kch@nvidia.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Bug: 265019048
Change-Id: I03cd1f6e36347dbe94ffc2aeb2ef50454a713864
(cherry picked from commit ef44c50837ab7818920bd9994b36d6e573312abc)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
From the UFSHCI 4.0 specification, about the legacy (single queue) mode:
"The host controller always process transfer requests in-order according
to the order submitted to the list. In case of multiple commands with
single doorbell register ringing (batch mode), The dispatch order for
these transfer requests by host controller will base on their index in
the List. A transfer request with lower index value will be executed
before a transfer request with higher index value."
From the UFSHCI 4.0 specification, about the MCQ mode:
"Command Submission
1. Host SW writes an Entry to SQ
2. Host SW updates SQ doorbell tail pointer
Command Processing
3. After fetching the Entry, Host Controller updates SQ doorbell head
pointer
4. Host controller sends COMMAND UPIU to UFS device"
In other words, for both legacy and MCQ mode, UFS controllers are
required to forward commands to the UFS device in the order these
commands have been received from the host.
Note: for legacy mode this is only correct if the host submits one
command at a time. The UFS driver does this.
Bug: 264714656
Change-Id: Ib123827a32828ee4dbe3c5e45d5fb83ffe5314da
Signed-off-by: Bart Van Assche <bvanassche@google.com>
From ZBC-2: "The device server terminates with CHECK CONDITION status, with
the sense key set to ILLEGAL REQUEST, and the additional sense code set to
UNALIGNED WRITE COMMAND a write command, other than an entire medium write
same command, that specifies: a) the starting LBA in a sequential write
required zone set to a value that is not equal to the write pointer for that
sequential write required zone; or b) an ending LBA that is not equal to the
last logical block within a physical block (see SBC-5)."
I am not aware of any other conditions that may trigger the UNALIGNED
WRITE COMMAND response.
Retry unaligned writes in preparation of removing zone locking.
Increase the number of retries for write commands sent to a sequential
zone to the maximum number of outstanding commands.
Bug: 264714656
Change-Id: If89c1f0b4d382978c52382dd3634f39fc15bcaf0
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Measurements have shown that limiting the queue depth to one for zoned
writes has a significant negative performance impact on zoned UFS devices.
Hence this patch that disables zone locking from the mq-deadline scheduler
for storage controllers that support pipelining zoned writes. This patch is
based on the following assumptions:
- Applications submit write requests to sequential write required zones
in order.
- It happens infrequently that zoned write requests are reordered by the
block layer.
- The storage controller does not reorder write requests that have been
submitted to the same hardware queue. This is the case for UFS: the
UFSHCI specification requires that UFS controllers process requests in
order per hardware queue.
- The I/O priority of all pipelined write requests is the same per zone.
- Either no I/O scheduler is used or an I/O scheduler is used that
submits write requests per zone in LBA order.
If applications submit write requests to sequential write required zones
in order, at least one of the pending requests will succeed. Hence, the
number of retries that is required is at most (number of pending
requests) - 1.
Bug: 264714656
Change-Id: I66265cd86e3f21e1a77ee7e49f94b0e756103c9b
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Writes in sequential write required zones must happen at the write
pointer. Even if the submitter of the write commands (e.g. a filesystem)
submits writes for sequential write required zones in order, the block
layer or the storage controller may reorder these write commands.
The zone locking mechanism in the mq-deadline I/O scheduler serializes
write commands for sequential zones. Some but not all storage controllers
require this serialization. Introduce a new flag such that block drivers
can request pipelining of writes for sequential write required zones.
An example of a storage controller standard that requires write
serialization is AHCI (Advanced Host Controller Interface). Submitting
commands to AHCI controllers happens by writing a bit pattern into a
register. Each set bit corresponds to an active command. This mechanism
does not preserve command ordering information.
Bug: 264714656
Change-Id: I1f64eb61571a66b99a0d83bef4438e2187c16f8e
Signed-off-by: Bart Van Assche <bvanassche@google.com>
process_notifier() is called every time a process exits. When multiple
processes exit roughly at the same time, the uid_lock taken from inside
of process_notifier() will create contention which slows down process
exit. Defer stats accounting in such case to avoid lock contention.
Bug: 261537194
Change-Id: Ia1e9a451eab39eb0dda7eb175bfd71c67f3e0a58
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
(cherry picked from commit 5d96c24be923d9011762de19bcfbade68b103759)
pKVM modules can't rely on the usual hyp function kern_hyp_va() to
convert addr from the kernel space to the hyp's. Instead, provide
pkvm_el2_mod_va() that will do the conversion using the token provided
by pkvm_load_el2_module().
Bug: 244543039
Bug: 244373730
Change-Id: I7423b40f1107bb92cd732843c5cdbf1d45662f00
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Exporsing HotPlugDetect(HPD) helps userspace to infer HPD
state as defined by VESA DisplayPort Alt Mode on USB Type-C Standard.
This allows userspace to notify users for self help, for instance,
to hint user that the display port cable is probably detached (or)
the display port sink (viz., monitors ect.,) is un-powered.
Also helps to debug issues reported from field.
This change adds an additional attribute "hpd" to the existing
"displayport" attributes.
VESA DisplayPort Alt Mode on USB Type-C Standard defines how
HotPlugDetect(HPD) shall be supported on the USB-C connector
when operating in DisplayPort Alt Mode. This is a read only
node which reflects the current state of HPD.
Valid values:
- 1 when HPD’s logical state is high (HPD_High)
- 0 when HPD’s logical state is low (HPD_Low)
This is also a partial backport from
7f8113948785 ("usb: typec: altmodes/displayport: Notify drm subsys of hotplug events")
as part of the logic is dependent on that change.
Bug: 253534975
Bug: 260915739
Change-Id: Id72e8ef6ede84038479649c2b753acdac547dea1
Signed-off-by: Badhri Jagan Sridharan <badhri@google.com>
Reviewed-by: Heikki Krogerus <heikki.krogerus@linux.intel.com>
Link: https://lore.kernel.org/lkml/20221211193755.1392128-1-badhri@google.com/T/
(cherry picked from commit 001b0c780eac328bc48b70b8437f202a4ed785e4
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb.git usb-testing)
Export find_vm_area for obtaining pages of vmalloc'ed memory, which is
required for both GXP and TPU modules.
Bug: 263839332
Change-Id: I1d6c37a5abb6012c3ff295120dd2d3cb2871c820
Signed-off-by: davidchiang <davidchiang@google.com>
Fix the calculation to determine the number of module relocs present in
the '.hyp.reloc' section to divide by the size of 'kvm_nvhe_reloc_t' (4)
instead of the size of a pointer (8).
Fixes: ace5025390 ("ANDROID: KVM: arm64: Resolve hyp module addresses using ELF sections")
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 261855285
Change-Id: Ia7afc508039d549ae061793afa39fde9d844c069
Modules with an empty '.hyp.text' section do not contain any EL2 code
and should therefore be ignored for the purposes of hypervisor module
loading. Failing to ignore such modules will likely result in a later
loading failure due to the absence of '.hyp.reloc', which is not present
for non-hypervisor modules.
Don't bother parsing the other '.hyp.*' sections for modules with an
empty '.hyp.text' section and return early success to allow the module
to load as a normal kernel module.
Fixes: ace5025390 ("ANDROID: KVM: arm64: Resolve hyp module addresses using ELF sections")
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 261855285
Change-Id: Idc24f95881c520b40038f77cd5af5ccc1d23624f
Introduce a function that makes it easy to verify whether a write
request is for a sequential write required or sequential write preferred
zone. Simplify blk_req_needs_zone_write_lock() by using the new function.
Bug: 264714656
Change-Id: I24677065f0c3b63900a885da5b4c8ea4ba0fb14c
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Since it is nontrivial to figure out how disk_zone_is_seq() and
blk_rq_zone_is_seq() handle sequential write preferred zones, document
this.
Bug: 264714656
Change-Id: I6d58683c302c29dc3134dcaf422b3e5c092092c1
Signed-off-by: Bart Van Assche <bvanassche@google.com>
Commit 237405ebef58 ("arm64: cpufeature: Force HWCAP to be based on the
sysreg visible to user-space") forced the hwcaps to use sanitised
user-space view of the id registers. However, the ID register structures
used to select few compat cpufeatures (vfp, crc32, ...) are masked and
hence such hwcaps do not appear in /proc/cpuinfo anymore for PER_LINUX32
personality.
Add the ID register structures explicitly and set the relevant entry as
visible. As these ID registers are now of type visible so make them
available in 64-bit userspace by making necessary changes in register
emulation logic and documentation.
While at it, update the comment for structure ftr_generic_32bits[] which
lists the ID register that use it.
Fixes: 237405ebef58 ("arm64: cpufeature: Force HWCAP to be based on the sysreg visible to user-space")
Cc: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: James Morse <james.morse@arm.com>
Signed-off-by: Amit Daniel Kachhap <amit.kachhap@arm.com>
Link: https://lore.kernel.org/r/20221103082232.19189-1-amit.kachhap@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Bug: 261510586
Change-Id: I9d1a779fe849e0a2069fc3e385c88532db84b7b4
(cherry picked from commit 85f1506337f0c79a4955edfeee86a18628e3735f)
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cortex-A510's erratum #2658417 causes two BF16 instructions to return the
wrong result in rare circumstances when a pair of A510 CPUs are using
shared neon hardware.
The two instructions affected are BFMMLA and VMMLA, support for these is
indicated by the BF16 HWCAP. Remove it on affected platforms.
Signed-off-by: James Morse <james.morse@arm.com>
Link: https://lore.kernel.org/r/20220909165938.3931307-4-james.morse@arm.com
[catalin.marinas@arm.com: add revision to the Kconfig help; remove .type]
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Conflicts:
arch/arm64/include/asm/sysreq.h
1. Added definition for ID_AA64ISAR1_EL1_BF16_MASK
Bug: 261510586
Change-Id: I83a5dd577fc8c0edd83c40b21f3fe54c54b6a9fa
(cherry picked from commit 1bdb0fbb2e27c51a6f311867726462c983a1d9ee)
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
get_arm64_ftr_reg() returns the properties of a system register based
on its instruction encoding.
This is needed by erratum workaround in cpu_errata.c to modify the
user-space visible view of id registers.
Signed-off-by: James Morse <james.morse@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20220909165938.3931307-3-james.morse@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Bug: 261510586
Change-Id: I637553fab833abfc349b391450c9f6a1173501a8
(cherry picked from commit 445c953e4a84b2942bdb5c0b5aff571c903680fe)
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
arm64 advertises hardware features to user-space via HWCAPs, and by
emulating access to the CPUs id registers. The cpufeature code has a
sanitised system-wide view of an id register, and a sanitised user-space
view of an id register, where some features use their 'safe' value
instead of the hardware value.
It is currently possible for a HWCAP to be advertised where the user-space
view of the id register does not show the feature as supported.
Erratum workaround need to remove both the HWCAP, and the feature from
the user-space view of the id register. This involves duplicating the
code, and spreading it over cpufeature.c and cpu_errata.c.
Make the HWCAP code use the user-space view of id registers. This ensures
the values never diverge, and allows erratum workaround to remove HWCAP
by modifying the user-space view of the id register.
Signed-off-by: James Morse <james.morse@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Link: https://lore.kernel.org/r/20220909165938.3931307-2-james.morse@arm.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
Bug: 261510586
Change-Id: I86bac3f501bf72ac9ee8fa6e9755ae9fbbdc81e6
(cherry picked from commit 237405ebef580a7352a52129b2465c117145eafa)
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Following "ANDROID: fips140: eliminate crypto-fips.a build step", the
only remaining dependency on LTO is the fact that the part of the module
linker script that merges the text and rodata sections and adds some
symbols is guarded by '#ifdef CONFIG_LTO_CLANG'.
This doesn't actually need to be the case, though. So guard it by
alternatively IS_ENABLED(CONFIG_CRYPTO_FIPS140_MOD).
Then, remove the dependency of CRYPTO_FIPS140_MOD on LTO_CLANG.
(Note: the android14-6.1 kernel currently has LTO disabled, which is
part of the motivation for this change. I don't know whether it will
stay that way, though.)
Bug: 188620248
Change-Id: I1aa7b293ac7a793721275e06e3ae40628e26bbc4
Signed-off-by: Eric Biggers <ebiggers@google.com>
To trick the build system into compiling some source files as built-in
code despite their actual destination being fips140.ko, a layer of
indirection was being used where the files were first built into a
static library crypto-fips.a, and then that static library was linked
into fips140.o before the final link of fips140.ko.
The problem with that approach is that it is incompatible with the usual
behavior of linking, where linking to a static library incorporates only
the needed parts of the library, not the whole library. The only reason
that it happened to work anyway is due to the dependency of the fips140
module on LTO, combined with a peculiarity of the way that the kernel
build system built LTO modules: the build system actually created
${modname}.o as a static library (despite the .o suffix), and used the
--whole-archive linker flag when linking ${modname}.ko.
commit c25e1c55822f ("kbuild: do not create *.prelink.o for Clang LTO or
IBT") in Linux v5.19 changed that. Now, ${modname}.o is an object file,
and the --whole-archive flag isn't used when linking ${modname}.ko.
Therefore, the crypto-fips.a hack no longer works, as things from this
static library (such as the initcalls) get lost during linking.
Replace it with a different hack that eliminates the dependency on LTO
and should be less fragile: undefine MODULE in fips140-defs.h, and
re-define it in the one file where it is needed. (For consistency, also
move the definition of __DISABLE_EXPORTS into fips140-defs.h.)
Bug: 188620248
Change-Id: I4a6a5f68381a7540bf37ba610216442dae0d2a7a
Signed-off-by: Eric Biggers <ebiggers@google.com>
KCFI generates ABS32 relocations in the .text section. In the temporary
copy of the .text section that the FIPS integrity check is done on,
these relocations need to be unapplied for the integrity check to pass.
Example from 'llvm-readelf --relocs --wide fips140.ko':
Relocation section '.rela.text' at offset 0x5b4a8 contains 2008 entries:
Offset Info Type Symbol's Value Symbol's Name + Addend
[...]
0000000000000c80 0000092900000102 R_AARCH64_ABS32 0000000050e29065 __kcfi_typeid_pmull_ghash_update_p64 + 0
0000000000000e08 0000092800000102 R_AARCH64_ABS32 0000000050e29065 __kcfi_typeid_pmull_ghash_update_p8 + 0
Bug: 188620248
Change-Id: I85c01641114a66b2603abce467977823469f50c8
Signed-off-by: Eric Biggers <ebiggers@google.com>
CONFIG_CRYPTO_FIPS140_MOD builds a loadable kernel module when set to
'y'. That's very unusual, as it doesn't follow the convention of
loadable modules being 'm'.
I'm guessing that the reason that a bool was used instead of a tristate
is because this functionality is not supported built-in, so there are
only two allowed settings: disabled or modular.
However, there's actually a way to express that in the kconfig language:
a tristate that depends on 'm'. Let's do it that way.
This also eliminates the need to explicitly depend on MODULES.
(Note: I decided to keep MOD in the name, since the word "module" in
"FIPS 140 cryptographic module" is a different meaning of "module", and
I didn't want to bother renaming CRYPTO_FIPS140_MOD_EVAL_TESTING too.)
Bug: 188620248
Change-Id: Ib195d64d68c23ca93dd244d9ac77255992870424
Signed-off-by: Eric Biggers <ebiggers@google.com>
There is no good reason for the CRYPTO_FIPS140 kconfig option to exist
separately from CRYPTO_FIPS140_MOD. It existed mainly to guard some
code in the module loader that was needed only for loading the fips140
module. However, that code has been removed, since an alternate
solution that doesn't require changes to the module loader was found.
The remaining references to CRYPTO_FIPS140 are in:
- scripts/module.lds.S. But the guarded code only affects building the
fips140 module, so CRYPTO_FIPS140_MOD should be used here instead.
- lib/crypto/, for guarding the Android vendor hooks required by the
fips140 module. However, Android vendor hooks are already guarded by
ANDROID_VENDOR_HOOKS. The extra guard by CRYPTO_FIPS140 isn't useful,
especially since CRYPTO_FIPS140 was effectively hardcoded to y anyway.
It did have the side effect of making the hooks be guarded by arm64,
which excluded them from builds of arch/x86/purgatory/. However, a
cleaner way to accomplish that is to check for __DISABLE_EXPORTS,
which handles both arch/x86/purgatory/ and fips140.ko itself.
Bug: 188620248
Change-Id: Ic6141cd2a553540c2bf95774e71de7310926e3ce
Signed-off-by: Eric Biggers <ebiggers@google.com>
Kernel code is expected to support both in-tree and out-of-tree builds.
Fix in-tree builds of the fips140 module by using the full include path
for fips140-defs.h.
Bug: 188620248
Change-Id: Iba5ed888bc6a316bd4d0a97479768f50094d63d0
Signed-off-by: Eric Biggers <ebiggers@google.com>
The customizations to the module linker script required by the fips140
module are currently split between scripts/module.lds.S and a file it
includes, arch/arm64/include/asm/module.lds.h. There isn't actually
anything architecture-specific about the part that was put into the
architecture-specific file, though. Let's move that part into
scripts/module.lds.S so that the fips140 changes are in the same file.
Bug: 188620248
Change-Id: If559a216e026d2107cdf3291b487686fdb21a5e8
Signed-off-by: Eric Biggers <ebiggers@google.com>
This reverts commit 9cb398bd05.
The '__hyprel_{start,end}' symbols are no longer used, so don't bother
generating them.
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 261855285
Change-Id: I8e8dc5c94a9e67400e73e362e4377032328d86d4
Resolving the addresses of the hypervisor sections within a loadable
module using symbol assignment is fragile, particularly in the face of
mergeable sections (i.e. those emitted with SHF_MERGE by the compiler).
Instead, parse the ELF .hyp.* sections directly and remove the need for
global symbols in the hypervisor module linker script.
Signed-off-by: Will Deacon <will@kernel.org>
[will: Preserve function_nocfi() calls which aren't needed in android14-6.1]
Signed-off-by: Will Deacon <willdeacon@google.com>
Bug: 261855285
Change-Id: I91d88e1a341b91ffe52ffc770dddc9b46ccb3aa4
Currently, only HS descriptors will be updated with endpoint address
during binding process. According to current max_speed in configfs,
this patch will also update SS/SSP descriptors with endpoint address.
Bug: 162562782
Signed-off-by: Ray Chi <raychi@google.com>
Change-Id: I67983ef47df7ac567ec1d3af80921c39c98a545d
(cherry picked from commit 41fe558317)
(cherry picked from commit ba3ec687b701c60f3b9336e6549b94ed84d8fe77)
There shouldn't be any restriction of the descriptor size (not the
descriptor id for that matter) up to QUERY_DESC_MAX_SIZE. According to the
spec, the caller can use any descriptor size, and it is up to the device to
return the actual size. Therefore there shouldn't be any sizes hardcoded
in the kernel, nor any need to cache it, hence the
ufshcd_map_desc_id_to_length() function is redundant. Always read the
descriptors with QUERY_DESC_MAX_SIZE size.
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Suggested-by: Bean Huo <beanhuo@micron.com>
Change-Id: I7933da0b7ec3771b9faccb2c8fe05bb83f741162
Signed-off-by: Arthur Simchaev <Arthur.Simchaev@wdc.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Bug: 258234315
(cherry picked from commit 16ed9d312b42180242476d159dd2fab05129029a git://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next)
Signed-off-by: Bart Van Assche <bvanassche@google.com>
