aboutsummaryrefslogtreecommitdiff
path: root/drivers/iommu/amd
AgeCommit message (Collapse)AuthorFilesLines
2022-10-21iommu: Add gfp parameter to iommu_alloc_resv_regionGravatar Lu Baolu 1-3/+4
Add gfp parameter to iommu_alloc_resv_region() for the callers to specify the memory allocation behavior. Thus iommu_alloc_resv_region() could also be available in critical contexts. Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Tested-by: Alex Williamson <alex.williamson@redhat.com> Link: https://lore.kernel.org/r/20220927053109.4053662-2-baolu.lu@linux.intel.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-26Merge branches 'apple/dart', 'arm/mediatek', 'arm/omap', 'arm/smmu', ↵Gravatar Joerg Roedel 9-120/+601
'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next
2022-09-26iommu/amd: Fix sparse warningGravatar Vasant Hegde 2-2/+2
CHECK drivers/iommu/amd/iommu.c drivers/iommu/amd/iommu.c:73:24: warning: symbol 'amd_iommu_ops' was not declared. Should it be static? Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220912063248.7909-6-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-26iommu/amd: Remove outdated commentGravatar Vasant Hegde 1-4/+0
Comment is not related to amd_iommu_ops variable. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220912063248.7909-5-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-26iommu/amd: Free domain ID after domain_flush_pagesGravatar Vasant Hegde 1-3/+3
free_io_pgtable_ops() path uses domain ID to flush pages. Hence free domain ID after flushing everything. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220912063248.7909-4-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-26iommu/amd: Free domain id in error pathGravatar Vasant Hegde 1-2/+6
Call domain_id_free() in error path. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220912063248.7909-2-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-11iommu: Fix false ownership failure on AMD systems with PASID activatedGravatar Jason Gunthorpe 1-0/+2
The AMD IOMMU driver cannot activate PASID mode on a RID without the RID's translation being set to IDENTITY. Further it requires changing the RID's page table layout from the normal v1 IOMMU_DOMAIN_IDENTITY layout to a different v2 layout. It does this by creating a new iommu_domain, configuring that domain for v2 identity operation and then attaching it to the group, from within the driver. This logic assumes the group is already set to the IDENTITY domain and is being used by the DMA API. However, since the ownership logic is based on the group's domain pointer equaling the default domain to detect DMA API ownership, this causes it to look like the group is not attached to the DMA API any more. This blocks attaching drivers to any other devices in the group. In a real system this manifests itself as the HD-audio devices on some AMD platforms losing their device drivers. Work around this unique behavior of the AMD driver by checking for equality of IDENTITY domains based on their type, not their pointer value. This allows the AMD driver to have two IDENTITY domains for internal purposes without breaking the check. Have the AMD driver properly declare that the special domain it created is actually an IDENTITY domain. Cc: Robin Murphy <robin.murphy@arm.com> Cc: stable@vger.kernel.org Fixes: 512881eacfa7 ("bus: platform,amba,fsl-mc,PCI: Add device DMA ownership management") Reported-by: Takashi Iwai <tiwai@suse.de> Tested-by: Takashi Iwai <tiwai@suse.de> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Robin Murphy <robin.murphy@arm.com> Link: https://lore.kernel.org/r/0-v1-ea566e16b06b+811-amd_owner_jgg@nvidia.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-09Merge branch 'iommu/fixes' into coreGravatar Joerg Roedel 1-1/+2
2022-09-09iommu/dma: Make header privateGravatar Robin Murphy 1-1/+1
Now that dma-iommu.h only contains internal interfaces, make it private to the IOMMU subsytem. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Link: https://lore.kernel.org/r/b237e06c56a101f77af142a54b629b27aa179d22.1660668998.git.robin.murphy@arm.com [ joro : re-add stub for iommu_dma_get_resv_regions ] Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Add command-line option to enable different page tableGravatar Vasant Hegde 1-5/+18
Enhance amd_iommu command line option to specify v1 or v2 page table. By default system will boot in V1 page table mode. Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-10-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Add support for using AMD IOMMU v2 page table for DMA-APIGravatar Suravee Suthikulpanit 1-0/+25
Introduce init function for setting up DMA domain for DMA-API with the IOMMU v2 page table. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-9-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Add support for Guest IO protectionGravatar Suravee Suthikulpanit 3-0/+19
AMD IOMMU introduces support for Guest I/O protection where the request from the I/O device without a PASID are treated as if they have PASID 0. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-8-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Initial support for AMD IOMMU v2 page tableGravatar Vasant Hegde 3-2/+420
Introduce IO page table framework support for AMD IOMMU v2 page table. This patch implements 4 level page table within iommu amd driver and supports 4K/2M/1G page sizes. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-7-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Update sanity check when enable PRI/ATS for IOMMU v1 tableGravatar Suravee Suthikulpanit 1-3/+11
Currently, PPR/ATS can be enabled only if the domain is type identity mapping. However, when allowing the IOMMU v2 page table to be used for DMA-API, the check is no longer valid. Update the sanity check to only apply for when using AMD_IOMMU_V1 page table mode. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-6-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Refactor amd_iommu_domain_enable_v2 to remove lockingGravatar Suravee Suthikulpanit 1-19/+27
The current function to enable IOMMU v2 also lock the domain. In order to reuse the same code in different code path, in which the domain has already been locked, refactor the function to separate the locking from the enabling logic. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-5-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Add map/unmap_pages() iommu_domain_ops callback supportGravatar Vasant Hegde 1-13/+16
Implement the map_pages() and unmap_pages() callback for the AMD IOMMU driver to allow calls from iommu core to map and unmap multiple pages. Also deprecate map/unmap callbacks. Finally gatherer is not updated by iommu_v1_unmap_pages(). Hence pass NULL instead of gather to iommu_v1_unmap_pages. Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-4-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd/io-pgtable: Implement unmap_pages io_pgtable_ops callbackGravatar Vasant Hegde 1-8/+9
Implement the io_pgtable_ops->unmap_pages() callback for AMD driver and deprecate io_pgtable_ops->unmap callback. Also if fetch_pte() returns NULL then return from unmap_mapages() instead of trying to continue to unmap remaining pages. Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-3-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd/io-pgtable: Implement map_pages io_pgtable_ops callbackGravatar Vasant Hegde 1-25/+34
Implement the io_pgtable_ops->map_pages() callback for AMD driver. Also deprecate io_pgtable->map callback. Suggested-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220825063939.8360-2-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/dma: Clean up KconfigGravatar Robin Murphy 1-1/+0
Although iommu-dma is a per-architecture chonce, that is currently implemented in a rather haphazard way. Selecting from the arch Kconfig was the original logical approach, but is complicated by having to manage dependencies; conversely, selecting from drivers ends up hiding the architecture dependency *too* well. Instead, let's just have it enable itself automatically when IOMMU API support is enabled for the relevant architectures. It can't get much clearer than that. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/2e33c8bc2b1bb478157b7964bfed976cb7466139.1660668998.git.robin.murphy@arm.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Clean up bus_set_iommu()Gravatar Robin Murphy 3-30/+1
Stop calling bus_set_iommu() since it's now unnecessary, and garbage-collect the last remnants of amd_iommu_init_api(). Signed-off-by: Robin Murphy <robin.murphy@arm.com> Link: https://lore.kernel.org/r/6bcc367e8802ae5a2b2840cbe4e9661ee024e80e.1660572783.git.robin.murphy@arm.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: Handle race between registration and device probeGravatar Robin Murphy 1-0/+4
As for the Intel driver, make sure the AMD driver can cope with seeing .probe_device calls without having to wait for all known instances to register first. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Link: https://lore.kernel.org/r/a8d8ebe12b411d28972f1ab928c6db92e8913cf5.1660572783.git.robin.murphy@arm.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu: Retire iommu_capable()Gravatar Robin Murphy 1-1/+1
With all callers now converted to the device-specific version, retire the old bus-based interface, and give drivers the chance to indicate accurate per-instance capabilities. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/d8bd8777d06929ad8f49df7fc80e1b9af32a41b5.1660574547.git.robin.murphy@arm.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-09-07iommu/amd: use full 64-bit value in build_completion_wait()Gravatar John Sperbeck 1-1/+2
We started using a 64 bit completion value. Unfortunately, we only stored the low 32-bits, so a very large completion value would never be matched in iommu_completion_wait(). Fixes: c69d89aff393 ("iommu/amd: Use 4K page for completion wait write-back semaphore") Signed-off-by: John Sperbeck <jsperbeck@google.com> Link: https://lore.kernel.org/r/20220801192229.3358786-1-jsperbeck@google.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-29Merge branches 'arm/exynos', 'arm/mediatek', 'arm/msm', 'arm/smmu', ↵Gravatar Joerg Roedel 7-675/+1133
'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next
2022-07-29iommu/amd: Fix compile warning in init codeGravatar Joerg Roedel 1-6/+3
A recent commit introduced these compile warnings: CC drivers/iommu/amd/init.o drivers/iommu/amd/init.c:938:12: error: ‘iommu_init_ga_log’ defined but not used [-Werror=unused-function] 938 | static int iommu_init_ga_log(struct amd_iommu *iommu) | ^~~~~~~~~~~~~~~~~ drivers/iommu/amd/init.c:902:12: error: ‘iommu_ga_log_enable’ defined but not used [-Werror=unused-function] 902 | static int iommu_ga_log_enable(struct amd_iommu *iommu) | ^~~~~~~~~~~~~~~~~~~ The warnings appear because both functions are defined when IRQ remapping is not enabled, but only used when IRQ remapping is enabled. Fix it by only defining the functions when IRQ remapping is enabled. Fixes: c5e1a1eb9279 ("iommu/amd: Simplify and Consolidate Virtual APIC (AVIC) Enablement") Signed-off-by: Joerg Roedel <jroedel@suse.de> Link: https://lore.kernel.org/r/20220729100432.22474-1-joro@8bytes.org
2022-07-29iommu/amd: Add support for AVIC when SNP is enabledGravatar Suravee Suthikulpanit 2-1/+17
In order to support AVIC on SNP-enabled system, The IOMMU driver needs to check EFR2[SNPAVICSup] and enables the support by setting SNPAVICEn bit in the IOMMU control register (MMIO offset 18h). For detail, please see section "SEV-SNP Guest Virtual APIC Support" of the AMD I/O Virtualization Technology (IOMMU) Specification. (https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf) Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com> Link: https://lore.kernel.org/r/20220726134348.6438-3-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-29iommu/amd: Simplify and Consolidate Virtual APIC (AVIC) EnablementGravatar Suravee Suthikulpanit 1-30/+55
Currently, enabling AVIC requires individually detect and enable GAM and GALOG features on each IOMMU, which is difficult to keep track on multi-IOMMU system, where the features needs to be enabled system-wide. In addition, these features do not need to be enabled in early stage. It can be delayed until after amd_iommu_init_pci(). Therefore, consolidate logic for detecting and enabling IOMMU GAM and GALOG features into a helper function, enable_iommus_vapic(), which uses the check_feature_on_all_iommus() helper function to ensure system-wide support of the features before enabling them, and postpone until after amd_iommu_init_pci(). The new function also check and clean up feature enablement residue from previous boot (e.g. in case of booting into kdump kernel), which triggers a WARN_ON (shown below) introduced by the commit a8d4a37d1bb9 ("iommu/amd: Restore GA log/tail pointer on host resume") in iommu_ga_log_enable(). [ 7.731955] ------------[ cut here ]------------ [ 7.736575] WARNING: CPU: 0 PID: 1 at drivers/iommu/amd/init.c:829 iommu_ga_log_enable.isra.0+0x16f/0x190 [ 7.746135] Modules linked in: [ 7.749193] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W -------- --- 5.19.0-0.rc7.53.eln120.x86_64 #1 [ 7.759706] Hardware name: Dell Inc. PowerEdge R7525/04D5GJ, BIOS 2.1.6 03/09/2021 [ 7.767274] RIP: 0010:iommu_ga_log_enable.isra.0+0x16f/0x190 [ 7.772931] Code: 20 20 00 00 8b 00 f6 c4 01 74 da 48 8b 44 24 08 65 48 2b 04 25 28 00 00 00 75 13 48 83 c4 10 5b 5d e9 f5 00 72 00 0f 0b eb e1 <0f> 0b eb dd e8 f8 66 42 00 48 8b 15 f1 85 53 01 e9 29 ff ff ff 48 [ 7.791679] RSP: 0018:ffffc90000107d20 EFLAGS: 00010206 [ 7.796905] RAX: ffffc90000780000 RBX: 0000000000000100 RCX: ffffc90000780000 [ 7.804038] RDX: 0000000000000001 RSI: ffffc90000780000 RDI: ffff8880451f9800 [ 7.811170] RBP: ffff8880451f9800 R08: ffffffffffffffff R09: 0000000000000000 [ 7.818303] R10: 0000000000000000 R11: 0000000000000000 R12: 0008000000000000 [ 7.825435] R13: ffff8880462ea900 R14: 0000000000000021 R15: 0000000000000000 [ 7.832572] FS: 0000000000000000(0000) GS:ffff888054a00000(0000) knlGS:0000000000000000 [ 7.840657] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7.846400] CR2: ffff888054dff000 CR3: 0000000053210000 CR4: 0000000000350eb0 [ 7.853533] Call Trace: [ 7.855979] <TASK> [ 7.858085] amd_iommu_enable_interrupts+0x180/0x270 [ 7.863051] ? iommu_setup+0x271/0x271 [ 7.866803] state_next+0x197/0x2c0 [ 7.870295] ? iommu_setup+0x271/0x271 [ 7.874049] iommu_go_to_state+0x24/0x2c [ 7.877976] amd_iommu_init+0xf/0x29 [ 7.881554] pci_iommu_init+0xe/0x36 [ 7.885133] do_one_initcall+0x44/0x200 [ 7.888975] do_initcalls+0xc8/0xe1 [ 7.892466] kernel_init_freeable+0x14c/0x199 [ 7.896826] ? rest_init+0xd0/0xd0 [ 7.900231] kernel_init+0x16/0x130 [ 7.903723] ret_from_fork+0x22/0x30 [ 7.907306] </TASK> [ 7.909497] ---[ end trace 0000000000000000 ]--- Fixes: commit a8d4a37d1bb9 ("iommu/amd: Restore GA log/tail pointer on host resume") Reported-by: Jerry Snitselaar <jsnitsel@redhat.com> Cc: Joerg Roedel <joro@8bytes.org> Cc: Maxim Levitsky <mlevitsk@redhat.com> Cc: Will Deacon <will@kernel.org> (maintainer:IOMMU DRIVERS) Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com> Link: https://lore.kernel.org/r/20220726134348.6438-2-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Do not support IOMMUv2 APIs when SNP is enabledGravatar Suravee Suthikulpanit 1-1/+6
The IOMMUv2 APIs (for supporting shared virtual memory with PASID) configures the domain with IOMMU v2 page table, and sets DTE[Mode]=0. This configuration cannot be supported on SNP-enabled system. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-10-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Do not support IOMMU_DOMAIN_IDENTITY after SNP is enabledGravatar Suravee Suthikulpanit 1-0/+7
Once SNP is enabled (by executing SNP_INIT command), IOMMU can no longer support the passthrough domain (i.e. IOMMU_DOMAIN_IDENTITY). The SNP_INIT command is called early in the boot process, and would fail if the kernel is configure to default to passthrough mode. After the system is already booted, users can try to change IOMMU domain type of a particular IOMMU group. In this case, the IOMMU driver needs to check the SNP-enable status and return failure when requesting to change domain type to identity. Therefore, return failure when trying to allocate identity domain. Reviewed-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-9-suravee.suthikulpanit@amd.com [ joro: Removed WARN_ON_ONCE() ] Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Set translation valid bit only when IO page tables are in useGravatar Suravee Suthikulpanit 2-3/+16
On AMD system with SNP enabled, IOMMU hardware checks the host translation valid (TV) and guest translation valid (GV) bits in the device table entry (DTE) before accessing the corresponded page tables. However, current IOMMU driver sets the TV bit for all devices regardless of whether the host page table is in use. This results in ILLEGAL_DEV_TABLE_ENTRY event for devices, which do not the host page table root pointer set up. Thefore, when SNP is enabled, only set TV bit when DMA remapping is not used, which is when domain ID in the AMD IOMMU device table entry (DTE) is zero. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-8-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Introduce function to check and enable SNPGravatar Brijesh Singh 2-0/+44
To support SNP, IOMMU needs to be enabled, and prohibits IOMMU configurations where DTE[Mode]=0, which means it cannot be supported with IOMMU passthrough domain (a.k.a IOMMU_DOMAIN_IDENTITY), and when AMD IOMMU driver is configured to not use the IOMMU host (v1) page table. Otherwise, RMP table initialization could cause the system to crash. The request to enable SNP support in IOMMU must be done before PCI initialization state of the IOMMU driver because enabling SNP affects how IOMMU driver sets up IOMMU data structures (i.e. DTE). Unlike other IOMMU features, SNP feature does not have an enable bit in the IOMMU control register. Instead, the IOMMU driver introduces an amd_iommu_snp_en variable to track enabling state of SNP. Introduce amd_iommu_snp_enable() for other drivers to request enabling the SNP support in IOMMU, which checks all prerequisites and determines if the feature can be safely enabled. Please see the IOMMU spec section 2.12 for further details. Reviewed-by: Robin Murphy <robin.murphy@arm.com> Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-7-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Globally detect SNP supportGravatar Suravee Suthikulpanit 1-2/+5
Modify existing SNP feature check to use the helper function check_feature_on_all_iommus() to ensure consistency among all IOMMUs. Also report IOMMU SNP support information for each IOMMU. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-6-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Process all IVHDs before enabling IOMMU featuresGravatar Suravee Suthikulpanit 1-6/+19
The ACPI IVRS table can contain multiple IVHD blocks. Each block contains information used to initialize each IOMMU instance. Currently, init_iommu_all sequentially process IVHD block and initialize IOMMU instance one-by-one. However, certain features require all IOMMUs to be configured in the same way system-wide. In case certain IVHD blocks contain inconsistent information (most likely FW bugs), the driver needs to go through and try to revert settings on IOMMUs that have already been configured. A solution is to split IOMMU initialization into 3 phases: Phase1 : Processes information of the IVRS table for all IOMMU instances. This allow all IVHDs to be processed prior to enabling features. Phase2 : Early feature support check on all IOMMUs (using information in IVHD blocks. Phase3 : Iterates through all IOMMU instances and enabling features. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-5-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Introduce global variable for storing common EFR and EFR2Gravatar Suravee Suthikulpanit 2-8/+40
Some IOMMU features require that all IOMMUs must support the feature, which is determined by checking the support bit in the Extended Feature Register 1 and 2 (EFR/EFR2) on all IOMMUs. This check is done by the function check_feature_on_all_iommus(), which iterates through all IOMMUs everytime it is called. Instead, introduce a global variable to store common EFR/EFR2 among all IOMMUs. In case of inconsistent EFR/EFR2 masks are detected on an IOMMU, a FW_BUG warning is reported. Suggested-by: Joerg Roedel <joro@8bytes.org> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-4-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Introduce Support for Extended Feature 2 RegisterGravatar Suravee Suthikulpanit 2-8/+20
AMD IOMMU spec introduces additional extended feature register in the IVRS IVHD offset 80h (for IVHD type 11h and 40h) and MMIO offset 1A0h. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-3-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu/amd: Change macro for IOMMU control register bit shift to decimal valueGravatar Suravee Suthikulpanit 1-21/+21
There is no functional change. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220713225651.20758-2-suravee.suthikulpanit@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-15iommu: remove the put_resv_regions methodGravatar Christoph Hellwig 1-1/+0
All drivers that implement get_resv_regions just use generic_put_resv_regions to implement the put side. Remove the indirections and document the allocations constraints. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Link: https://lore.kernel.org/r/20220708080616.238833-4-hch@lst.de Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Update amd_iommu_fault structure to include PCI seg IDGravatar Vasant Hegde 3-6/+7
Rename 'device_id' as 'sbdf' and extend it to 32bit so that we can pass PCI segment ID to ppr_notifier(). Also pass PCI segment ID to pci_get_domain_bus_and_slot() instead of default value. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-36-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Update device_state structure to include PCI seg IDGravatar Vasant Hegde 1-34/+24
Rename struct device_state.devid variable to struct device_state.sbdf and extend it to 32-bit to include the 16-bit PCI segment ID via the helper function get_pci_sbdf_id(). Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-35-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Print PCI segment ID in error log messagesGravatar Vasant Hegde 2-23/+23
Print pci segment ID along with bdf. Useful for debugging. Co-developed-by: Suravee Suthikulpaint <suravee.suthikulpanit@amd.com> Signed-off-by: Suravee Suthikulpaint <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-34-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Add PCI segment support for ivrs_[ioapic/hpet/acpihid] commandsGravatar Suravee Suthikulpanit 1-17/+27
By default, PCI segment is zero and can be omitted. To support system with non-zero PCI segment ID, modify the parsing functions to allow PCI segment ID. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-33-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Specify PCI segment ID when getting pci deviceGravatar Suravee Suthikulpanit 2-11/+14
Upcoming AMD systems can have multiple PCI segments. Hence pass PCI segment ID to pci_get_domain_bus_and_slot() instead of '0'. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-32-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Include PCI segment ID when initialize IOMMUGravatar Suravee Suthikulpanit 4-33/+35
Extend current device ID variables to 32-bit to include the 16-bit segment ID when parsing device information from IVRS table to initialize each IOMMU. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-31-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Introduce get_device_sbdf_id() helper functionGravatar Suravee Suthikulpanit 3-29/+38
Current get_device_id() only provide 16-bit PCI device ID (i.e. BDF). With multiple PCI segment support, we need to extend the helper function to include PCI segment ID. So, introduce a new helper function get_device_sbdf_id() to replace the current get_pci_device_id(). Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-30-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Flush upto last_bdf onlyGravatar Vasant Hegde 1-3/+6
Fix amd_iommu_flush_dte_all() and amd_iommu_flush_tlb_all() to flush upto last_bdf only. Co-developed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-29-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Remove global amd_iommu_[dev_table/alias_table/last_bdf]Gravatar Suravee Suthikulpanit 3-95/+27
Replace them with per PCI segment device table. Also remove dev_table_size, alias_table_size, amd_iommu_last_bdf variables. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-28-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Update set_dev_entry_bit() and get_dev_entry_bit()Gravatar Suravee Suthikulpanit 3-22/+41
To include a pointer to per PCI segment device table. Also include struct amd_iommu as one of the function parameter to amd_iommu_apply_erratum_63() since it is needed when setting up DTE. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-27-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Update (un)init_device_table_dma()Gravatar Suravee Suthikulpanit 1-7/+20
Include struct amd_iommu_pci_seg as a function parameter since we need to access per PCI segment device table. Co-developed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-26-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Update set_dte_irq_entryGravatar Suravee Suthikulpanit 1-6/+7
Start using per PCI segment device table instead of global device table. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-25-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
2022-07-07iommu/amd: Update dump_dte_entryGravatar Suravee Suthikulpanit 1-4/+4
Start using per PCI segment device table instead of global device table. Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20220706113825.25582-24-vasant.hegde@amd.com Signed-off-by: Joerg Roedel <jroedel@suse.de>