Peter Maydell [Sat, 22 Dec 2018 11:25:31 +0000 (11:25 +0000)]
Merge remote-tracking branch 'remotes/marcel/tags/rdma-pull-request' into staging
RDMA queue
* Add support for RDMA MAD
* Various fixes for the pvrdma backend
# gpg: Signature made Sat 22 Dec 2018 09:36:36 GMT
# gpg: using RSA key 36D4C0F0CF2FE46D
# gpg: Good signature from "Marcel Apfelbaum <[email protected]>"
# gpg: aka "Marcel Apfelbaum <[email protected]>"
# gpg: aka "Marcel Apfelbaum <[email protected]>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: B1C6 3A57 F92E 08F2 640F 31F5 36D4 C0F0 CF2F E46D
* remotes/marcel/tags/rdma-pull-request: (31 commits)
pvrdma: check return value from pvrdma_idx_ring_has_ routines
rdma: remove unused VENDOR_ERR_NO_SGE macro
pvrdma: release ring object in case of an error
pvrdma: check number of pages when creating rings
pvrdma: add uar_read routine
rdma: check num_sge does not exceed MAX_SGE
pvrdma: release device resources in case of an error
docs: Update pvrdma device documentation
hw/rdma: Do not call rdma_backend_del_gid on an empty gid
hw/rdma: Do not use bitmap_zero_extend to free bitmap
hw/pvrdma: Clean device's resource when system is shutdown
vl: Introduce shutdown_notifiers
hw/rdma: Remove unneeded code that handles more that one port
hw/pvrdma: Fill error code in command's response
hw/pvrdma: Fill all CQE fields
hw/pvrdma: Make device state depend on Ethernet function state
hw/rdma: Initialize node_guid from vmxnet3 mac address
hw/pvrdma: Make sure PCI function 0 is vmxnet3
vmxnet3: Move some definitions to header file
hw/pvrdma: Add support to allow guest to configure GID table
...
pvrdma: check return value from pvrdma_idx_ring_has_ routines
pvrdma_idx_ring_has_[data/space] routines also return invalid
index PVRDMA_INVALID_IDX[=-1], if ring has no data/space. Check
return value from these routines to avoid plausible infinite loops.
When creating CQ/QP rings, an object can have up to
PVRDMA_MAX_FAST_REG_PAGES 8 pages. Check 'npages' parameter
to avoid excessive memory allocation or a null dereference.
rdma back-end has scatter/gather array ibv_sge[MAX_SGE=4] set
to have 4 elements. A guest could send a 'PvrdmaSqWqe' ring element
with 'num_sge' set to > MAX_SGE, which may lead to OOB access issue.
Add check to avoid it.
Yuval Shaia [Fri, 21 Dec 2018 14:40:37 +0000 (16:40 +0200)]
docs: Update pvrdma device documentation
Interface with the device is changed with the addition of support for
MAD packets.
Adjust documentation accordingly.
While there fix a minor mistake which may lead to think that there is a
relation between using RXE on host and the compatibility with bare-metal
peers.
Yuval Shaia [Fri, 21 Dec 2018 14:40:36 +0000 (16:40 +0200)]
hw/rdma: Do not call rdma_backend_del_gid on an empty gid
When device goes down the function fini_ports loops over all entries in
gid table regardless of the fact whether entry is valid or not. In case
that entry is not valid we'd like to skip from any further processing in
backend device.
Yuval Shaia [Fri, 21 Dec 2018 14:40:33 +0000 (16:40 +0200)]
vl: Introduce shutdown_notifiers
Notifier will be used for signaling shutdown event to inform system is
shutdown. This will allow devices and other component to run some
cleanup code needed before VM is shutdown.
Yuval Shaia [Fri, 21 Dec 2018 14:40:31 +0000 (16:40 +0200)]
hw/pvrdma: Fill error code in command's response
Driver checks error code let's set it.
In addition, for code simplification purposes, set response's fields
ack, response and err outside of the scope of command handlers.
Yuval Shaia [Fri, 21 Dec 2018 14:40:29 +0000 (16:40 +0200)]
hw/pvrdma: Make device state depend on Ethernet function state
User should be able to control the device by changing Ethernet function
state so if user runs 'ifconfig ens3 down' the PVRDMA function should be
down as well.
Yuval Shaia [Fri, 21 Dec 2018 14:40:26 +0000 (16:40 +0200)]
vmxnet3: Move some definitions to header file
pvrdma setup requires vmxnet3 device on PCI function 0 and PVRDMA device
on PCI function 1.
pvrdma device needs to access vmxnet3 device object for several reasons:
1. Make sure PCI function 0 is vmxnet3.
2. To monitor vmxnet3 device state.
3. To configure node_guid accoring to vmxnet3 device's MAC address.
To be able to access vmxnet3 device the definition of VMXNET3State is
moved to a new header file.
Yuval Shaia [Fri, 21 Dec 2018 14:40:25 +0000 (16:40 +0200)]
hw/pvrdma: Add support to allow guest to configure GID table
The control over the RDMA device's GID table is done by updating the
device's Ethernet function addresses.
Usually the first GID entry is determined by the MAC address, the second
by the first IPv6 address and the third by the IPv4 address. Other
entries can be added by adding more IP addresses. The opposite is the
same, i.e. whenever an address is removed, the corresponding GID entry
is removed.
The process is done by the network and RDMA stacks. Whenever an address
is added the ib_core driver is notified and calls the device driver
add_gid function which in turn update the device.
To support this in pvrdma device we need to hook into the create_bind
and destroy_bind HW commands triggered by pvrdma driver in guest.
Whenever a change is made to the pvrdma port's GID table a special QMP
message is sent to be processed by libvirt to update the address of the
backend Ethernet device.
Yuval Shaia [Fri, 21 Dec 2018 14:40:24 +0000 (16:40 +0200)]
qapi: Define new QMP message for pvrdma
pvrdma requires that the same GID attached to it will be attached to the
backend device in the host.
A new QMP messages is defined so pvrdma device can broadcast any change
made to its GID table. This event is captured by libvirt which in turn
will update the GID table in the backend device.
Yuval Shaia [Fri, 21 Dec 2018 14:40:22 +0000 (16:40 +0200)]
hw/pvrdma: Set the correct opcode for recv completion
The function pvrdma_post_cqe populates CQE entry with opcode from the
given completion element. For receive operation value was not set. Fix
it by setting it to IBV_WC_RECV.
Yuval Shaia [Fri, 21 Dec 2018 14:40:19 +0000 (16:40 +0200)]
hw/rdma: Add support for MAD packets
MAD (Management Datagram) packets are widely used by various modules
both in kernel and in user space for example the rdma_* API which is
used to create and maintain "connection" layer on top of RDMA uses
several types of MAD packets.
For more information please refer to chapter 13.4 in Volume 1
Architecture Specification, Release 1.1 available here:
https://www.infinibandta.org/ibta-specifications-download/
To support MAD packets the device uses an external utility
(contrib/rdmacm-mux) to relay packets from and to the guest driver.
Yuval Shaia [Fri, 21 Dec 2018 14:40:16 +0000 (16:40 +0200)]
hw/rdma: Add ability to force notification without re-arm
Upon completion of incoming packet the device pushes CQE to driver's RX
ring and notify the driver (msix).
While for data-path incoming packets the driver needs the ability to
control whether it wished to receive interrupts or not, for control-path
packets such as incoming MAD the driver needs to be notified anyway, it
even do not need to re-arm the notification bit.
Yuval Shaia [Fri, 21 Dec 2018 14:40:15 +0000 (16:40 +0200)]
contrib/rdmacm-mux: Add implementation of RDMA User MAD multiplexer
RDMA MAD kernel module (ibcm) disallow more than one MAD-agent for a
given MAD class.
This does not go hand-by-hand with qemu pvrdma device's requirements
where each VM is MAD agent.
Fix it by adding implementation of RDMA MAD multiplexer service which on
one hand register as a sole MAD agent with the kernel module and on the
other hand gives service to more than one VM.
Design Overview: Reviewed-by: Shamir Rabinovitch <[email protected]>
----------------
A server process is registered to UMAD framework (for this to work the
rdma_cm kernel module needs to be unloaded) and creates a unix socket to
listen to incoming request from clients.
A client process (such as QEMU) connects to this unix socket and
registers with its own GID.
TX:
----
When client needs to send rdma_cm MAD message it construct it the same
way as without this multiplexer, i.e. creates a umad packet but this
time it writes its content to the socket instead of calling umad_send().
The server, upon receiving such a message fetch local_comm_id from it so
a context for this session can be maintain and relay the message to UMAD
layer by calling umad_send().
RX:
----
The server creates a worker thread to process incoming rdma_cm MAD
messages. When an incoming message arrived (umad_recv()) the server,
depending on the message type (attr_id) looks for target client by
either searching in gid->fd table or in local_comm_id->fd table. With
the extracted fd the server relays to incoming message to the client.
Peter Maydell [Fri, 21 Dec 2018 15:49:59 +0000 (15:49 +0000)]
Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-4.0-20181221' into staging
ppc patch queue 2018-12-21
This pull request supersedes the one from 2018-12-13.
This is a revised first ppc pull request for qemu-4.0. Highlights
are:
* Most of the code for the POWER9 "XIVE" interrupt controller
(not complete yet, but we're getting there)
* A number of g_new vs. g_malloc cleanups
* Some IRQ wiring cleanups
* A fix for how we advertise NUMA nodes to the guest for pseries
* remotes/dgibson/tags/ppc-for-4.0-20181221: (40 commits)
MAINTAINERS: PPC: add a XIVE section
spapr: change default CPU type to POWER9
spapr: introduce an 'ic-mode' machine option
spapr: add an extra OV5 field to the sPAPR IRQ backend
spapr: add a 'reset' method to the sPAPR IRQ backend
spapr: extend the sPAPR IRQ backend for XICS migration
spapr: allocate the interrupt thread context under the CPU core
spapr: add device tree support for the XIVE exploitation mode
spapr: add hcalls support for the XIVE exploitation interrupt mode
spapr: introduce a new machine IRQ backend for XIVE
spapr-iommu: Always advertise the maximum possible DMA window size
spapr/xive: use the VCPU id as a NVT identifier
spapr/xive: introduce a XIVE interrupt controller
ppc/xive: notify the CPU when the interrupt priority is more privileged
ppc/xive: introduce a simplified XIVE presenter
ppc/xive: introduce the XIVE interrupt thread context
ppc/xive: add support for the END Event State Buffers
Changes requirement for "vsubsbs" instruction
spapr: export and rename the xics_max_server_number() routine
spapr: introduce a spapr_irq_init() routine
...
Peter Maydell [Fri, 21 Dec 2018 14:06:01 +0000 (14:06 +0000)]
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging
pci, pc, virtio: fixes, features
VTD fixes
IR and split irqchip are now the default for Q35
ACPI refactoring
hotplug refactoring
new names for virtio devices
multiple pcie link width/speeds
PCI fixes
Signed-off-by: Michael S. Tsirkin <[email protected]>
# gpg: Signature made Thu 20 Dec 2018 18:26:03 GMT
# gpg: using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <[email protected]>"
# gpg: aka "Michael S. Tsirkin <[email protected]>"
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469
* remotes/mst/tags/for_upstream: (44 commits)
x86-iommu: turn on IR by default if proper
x86-iommu: switch intr_supported to OnOffAuto type
q35: set split kernel irqchip as default
pci: Adjust PCI config limit based on bus topology
spapr_pci: perform unplug via the hotplug handler
pci/shpc: perform unplug via the hotplug handler
pci: Reuse pci-bridge hotplug handler handlers for pcie-pci-bridge
pci/pcie: perform unplug via the hotplug handler
pci/pcihp: perform unplug via the hotplug handler
pci/pcihp: overwrite hotplug handler recursively from the start
pci/pcihp: perform check for bus capability in pre_plug handler
s390x/pci: rename hotplug handler callbacks
pci/shpc: rename hotplug handler callbacks
pci/pcie: rename hotplug handler callbacks
hw/i386: Remove deprecated machines pc-0.10 and pc-0.11
hw: acpi: Remove AcpiRsdpDescriptor and fix tests
hw: acpi: Export and share the ARM RSDP build
hw: arm: Support both legacy and current RSDP build
hw: arm: Convert the RSDP build to the buid_append_foo() API
hw: arm: Carry RSDP specific data through AcpiRsdpData
...
Cédric Le Goater [Mon, 17 Dec 2018 22:34:43 +0000 (23:34 +0100)]
spapr: introduce an 'ic-mode' machine option
This option is used to select the interrupt controller mode (XICS or
XIVE) with which the machine will operate. XICS being the default
mode for now.
When running a machine with the XIVE interrupt mode backend, the guest
OS is required to have support for the XIVE exploitation mode. In the
case of legacy OS, the mode selected by CAS should be XICS and the OS
should fail to boot. However, QEMU could possibly detect it, terminate
the boot process and reset to stop in the SLOF firmware. This is not
yet handled.
Cédric Le Goater [Mon, 17 Dec 2018 22:34:42 +0000 (23:34 +0100)]
spapr: add an extra OV5 field to the sPAPR IRQ backend
The interrupt modes supported by the hypervisor are advertised to the
guest with new bits definitions of the option vector 5 of property
"ibm,arch-vec-5-platform-support. The byte 23 bits 0-1 of the OV5 are
defined as follow :
0b00 PAPR 2.7 and earlier (Legacy systems)
0b01 XIVE Exploitation mode only
0b10 Either available
If the client/guest selects the XIVE interrupt mode, it informs the
hypervisor by returning the value 0b01 in byte 23 bits 0-1. A 0b00
value indicates the use of the XICS interrupt mode (Legacy systems).
The sPAPR IRQ backend is extended with these definitions and the
values are directly used to populate the "ibm,arch-vec-5-platform-support"
property. The interrupt mode is advertised under TCG and under KVM.
Although a KVM XIVE device is not yet available, the machine can still
operate with kernel_irqchip=off. However, we apply a restriction on
the CPU which is required to be a POWER9 when a XIVE interrupt
controller is in use.
Cédric Le Goater [Tue, 11 Dec 2018 22:38:17 +0000 (23:38 +0100)]
spapr: add a 'reset' method to the sPAPR IRQ backend
For the time being, the XIVE reset handler updates the OS CAM line of
the vCPU as it is done under a real hypervisor when a vCPU is
scheduled to run on a HW thread. This will let the XIVE presenter
engine find a match among the NVTs dispatched on the HW threads.
This handler will become even more useful when we introduce the
machine supporting both interrupt modes, XIVE and XICS. In this
machine, the interrupt mode is chosen by the CAS negotiation process
and activated after a reset.
Cédric Le Goater [Tue, 11 Dec 2018 22:38:14 +0000 (23:38 +0100)]
spapr: add device tree support for the XIVE exploitation mode
The XIVE interface for the guest is described in the device tree under
the "interrupt-controller" node. A couple of new properties are
specific to XIVE :
- "reg"
contains the base address and size of the thread interrupt
managnement areas (TIMA), for the User level and for the Guest OS
level. Only the Guest OS level is taken into account today.
- "ibm,xive-eq-sizes"
the size of the event queues. One cell per size supported, contains
log2 of size, in ascending order.
- "ibm,xive-lisn-ranges"
the IRQ interrupt number ranges assigned to the guest for the IPIs.
and also under the root node :
- "ibm,plat-res-int-priorities"
contains a list of priorities that the hypervisor has reserved for
its own use. OPAL uses the priority 7 queue to automatically
escalate interrupts for all other queues (DD2.X POWER9). So only
priorities [0..6] are allowed for the guest.
Extend the sPAPR IRQ backend with a new handler to populate the DT
with the appropriate "interrupt-controller" node.
Cédric Le Goater [Tue, 11 Dec 2018 22:38:13 +0000 (23:38 +0100)]
spapr: add hcalls support for the XIVE exploitation interrupt mode
The different XIVE virtualization structures (sources and event queues)
are configured with a set of Hypervisor calls :
- H_INT_GET_SOURCE_INFO
used to obtain the address of the MMIO page of the Event State
Buffer (ESB) entry associated with the source.
- H_INT_SET_SOURCE_CONFIG
assigns a source to a "target".
- H_INT_GET_SOURCE_CONFIG
determines which "target" and "priority" is assigned to a source
- H_INT_GET_QUEUE_INFO
returns the address of the notification management page associated
with the specified "target" and "priority".
- H_INT_SET_QUEUE_CONFIG
sets or resets the event queue for a given "target" and "priority".
It is also used to set the notification configuration associated
with the queue, only unconditional notification is supported for
the moment. Reset is performed with a queue size of 0 and queueing
is disabled in that case.
- H_INT_GET_QUEUE_CONFIG
returns the queue settings for a given "target" and "priority".
- H_INT_RESET
resets all of the guest's internal interrupt structures to their
initial state, losing all configuration set via the hcalls
H_INT_SET_SOURCE_CONFIG and H_INT_SET_QUEUE_CONFIG.
- H_INT_SYNC
issue a synchronisation on a source to make sure all notifications
have reached their queue.
Cédric Le Goater [Tue, 11 Dec 2018 22:38:12 +0000 (23:38 +0100)]
spapr: introduce a new machine IRQ backend for XIVE
The XIVE IRQ backend uses the same layout as the new XICS backend but
covers the full range of the IRQ number space. The IRQ numbers for the
CPU IPIs are allocated at the bottom of this space, below 4K, to
preserve compatibility with XICS which does not use that range.
This should be enough given that the maximum number of CPUs is 1024
for the sPAPR machine under QEMU. For the record, the biggest POWER8
or POWER9 system has a maximum of 1536 HW threads (16 sockets, 192
cores, SMT8).
spapr-iommu: Always advertise the maximum possible DMA window size
When deciding about the huge DMA window, the typical Linux pseries guest
uses the maximum allowed RAM size as the upper limit. We did the same
on QEMU side to match that logic. Now we are going to support a GPU RAM
pass through which is not available at the guest boot time as it requires
the guest driver interaction. As the result, the guest requests a smaller
window than it should. Therefore the guest needs to be patched to
understand this new memory and so does QEMU.
Instead of reimplementing here whatever solution we choose for the guest,
this advertises the biggest possible window size limited by 32 bit
(as defined by LoPAPR). Since the window size has to be power-of-two
(the create rtas call receives a window shift, not a size),
this uses 0x8000.0000 as the maximum number of TCEs possible (rather than
32bit maximum of 0xffff.ffff).
This is safe as:
1. The guest visible emulated table is allocated in KVM (actual pages
are allocated in page fault handler) and QEMU (actual pages are allocated
when updated);
2. The hardware table (and corresponding userspace address table)
supports sparse allocation and also checks for locked_vm limit so
it is unable to cause the host any damage.
The IVPE scans the O/S CAM line of the XIVE thread interrupt contexts
to find a matching Notification Virtual Target (NVT) among the NVTs
dispatched on the HW processor threads.
On a real system, the thread interrupt contexts are updated by the
hypervisor when a Virtual Processor is scheduled to run on a HW
thread. Under QEMU, the model will emulate the same behavior by
hardwiring the NVT identifier in the thread context registers at
reset.
The NVT identifier used by the sPAPRXive model is the VCPU id. The END
identifier is also derived from the VCPU id. A set of helpers doing
the conversion between identifiers are provided for the hcalls
configuring the sources and the ENDs.
The model does not need a NVT table but the XiveRouter NVT operations
are provided to perform some extra checks in the routing algorithm.
The sPAPRXive model incorporates an internal XiveSource for the IPIs
and for the interrupts of the virtual devices of the guest. This model
is consistent with XIVE architecture which also incorporates an
internal IVSE for IPIs and accelerator interrupts in the IVRE
sub-engine.
The sPAPRXive model exports two memory regions, one for the ESB
trigger and management pages used to control the sources and one for
the TIMA pages. They are mapped by default at the addresses found on
chip 0 of a baremetal system. This is also consistent with the XIVE
architecture which defines a Virtualization Controller BAR for the
internal IVSE ESB pages and a Thread Managment BAR for the TIMA.
ppc/xive: notify the CPU when the interrupt priority is more privileged
After the event data was enqueued in the O/S Event Queue, the IVPE
raises the bit corresponding to the priority of the pending interrupt
in the register IBP (Interrupt Pending Buffer) to indicate there is an
event pending in one of the 8 priority queues. The Pending Interrupt
Priority Register (PIPR) is also updated using the IPB. This register
represent the priority of the most favored pending notification.
The PIPR is then compared to the the Current Processor Priority
Register (CPPR). If it is more favored (numerically less than), the
CPU interrupt line is raised and the EO bit of the Notification Source
Register (NSR) is updated to notify the presence of an exception for
the O/S. The check needs to be done whenever the PIPR or the CPPR are
changed.
The O/S acknowledges the interrupt with a special load in the Thread
Interrupt Management Area. If the EO bit of the NSR is set, the CPPR
takes the value of PIPR. The bit number in the IBP corresponding to
the priority of the pending interrupt is reseted and so is the EO bit
of the NSR.
The last sub-engine of the XIVE architecture is the Interrupt
Virtualization Presentation Engine (IVPE). On HW, the IVRE and the
IVPE share elements, the Power Bus interface (CQ), the routing table
descriptors, and they can be combined in the same HW logic. We do the
same in QEMU and combine both engines in the XiveRouter for
simplicity.
When the IVRE has completed its job of matching an event source with a
Notification Virtual Target (NVT) to notify, it forwards the event
notification to the IVPE sub-engine. The IVPE scans the thread
interrupt contexts of the Notification Virtual Targets (NVT)
dispatched on the HW processor threads and if a match is found, it
signals the thread. If not, the IVPE escalates the notification to
some other targets and records the notification in a backlog queue.
The IVPE maintains the thread interrupt context state for each of its
NVTs not dispatched on HW processor threads in the Notification
Virtual Target table (NVTT).
The model currently only supports single NVT notifications.
Signed-off-by: Cédric Le Goater <[email protected]>
[dwg: Folded in fix for field accessors] Signed-off-by: David Gibson <[email protected]>
Each exception has a state independent from the others called a Thread
Interrupt Management context. This context is a set of registers which
lets the thread handle priority management and interrupt acknowledgment
among other things. The most important ones being :
These registers are accessible through a specific MMIO region, called
the Thread Interrupt Management Area (TIMA), four aligned pages, each
exposing a different view of the registers. First page (page address
ending in 0b00) gives access to the entire context and is reserved for
the ring 0 view for the physical thread context. The second (page
address ending in 0b01) is for the hypervisor, ring 1 view. The third
(page address ending in 0b10) is for the operating system, ring 2
view. The fourth (page address ending in 0b11) is for user level, ring
3 view.
The thread interrupt context is modeled with a XiveTCTX object
containing the values of the different exception registers. The TIMA
region is mapped at the same address for each CPU.
ppc/xive: add support for the END Event State Buffers
The Event Notification Descriptor (END) XIVE structure also contains
two Event State Buffers providing further coalescing of interrupts,
one for the notification event (ESn) and one for the escalation events
(ESe). A MMIO page is assigned for each to control the EOI through
loads only. Stores are not allowed.
The END ESBs are modeled through an object resembling the 'XiveSource'
It is stateless as the END state bits are backed into the XiveEND
structure under the XiveRouter and the MMIO accesses follow the same
rules as for the XiveSource ESBs.
END ESBs are not supported by the Linux drivers neither on OPAL nor on
sPAPR. Nevetherless, it provides a mean to study the question in the
future and validates a bit more the XIVE model.
Signed-off-by: Cédric Le Goater <[email protected]>
[dwg: Fold in a later fix for field access] Signed-off-by: David Gibson <[email protected]>
spapr: initialize VSMT before initializing the IRQ backend
We will need to use xics_max_server_number() to create the sPAPRXive
object modeling the interrupt controller of the machine which is
created before the CPUs.
ppc/xive: introduce the XIVE Event Notification Descriptors
To complete the event routing, the IVRE sub-engine uses a second table
containing Event Notification Descriptor (END) structures.
An END specifies on which Event Queue (EQ) the event notification
data, defined in the associated EAS, should be posted when an
exception occurs. It also defines which Notification Virtual Target
(NVT) should be notified.
The Event Queue is a memory page provided by the O/S defining a
circular buffer, one per server and priority couple, containing Event
Queue entries. These are 4 bytes long, the first bit being a
'generation' bit and the 31 following bits the END Data field. They
are pulled by the O/S when the exception occurs.
The END Data field is a way to set an invariant logical event source
number for an IRQ. On sPAPR machines, it is set with the
H_INT_SET_SOURCE_CONFIG hcall when the EISN flag is used.
Signed-off-by: Cédric Le Goater <[email protected]>
[dwg: Fold in a later fix from Cédric fixing field accessors] Signed-off-by: David Gibson <[email protected]>
The XiveRouter models the second sub-engine of the XIVE architecture :
the Interrupt Virtualization Routing Engine (IVRE).
The IVRE handles event notifications of the IVSE and performs the
interrupt routing process. For this purpose, it uses a set of tables
stored in system memory, the first of which being the Event Assignment
Structure (EAS) table.
The EAT associates an interrupt source number with an Event Notification
Descriptor (END) which will be used in a second phase of the routing
process to identify a Notification Virtual Target.
The XiveRouter is an abstract class which needs to be inherited from
to define a storage for the EAT, and other upcoming tables.
Signed-off-by: Cédric Le Goater <[email protected]>
[dwg: Folded in parts of a later fix by Cédric fixing field access]
[dwg: Fix style nits] Signed-off-by: David Gibson <[email protected]>
The XiveNotifier offers a simple interface, between the XiveSource
object and the main interrupt controller of the machine. It will
forward event notifications to the XIVE Interrupt Virtualization
Routing Engine (IVRE).
Signed-off-by: Cédric Le Goater <[email protected]>
[dwg: Adjust type name string for XiveNotifier] Signed-off-by: David Gibson <[email protected]>
ppc/xive: add support for the LSI interrupt sources
The 'sent' status of the LSI interrupt source is modeled with the 'P'
bit of the ESB and the assertion status of the source is maintained
with an extra bit under the main XiveSource object. The type of the
source is stored in the same array for practical reasons.
The first sub-engine of the overall XIVE architecture is the Interrupt
Virtualization Source Engine (IVSE). An IVSE can be integrated into
another logic, like in a PCI PHB or in the main interrupt controller
to manage IPIs.
Each IVSE instance is associated with an Event State Buffer (ESB) that
contains a two bit state entry for each possible event source. When an
event is signaled to the IVSE, by MMIO or some other means, the
associated interrupt state bits are fetched from the ESB and
modified. Depending on the resulting ESB state, the event is forwarded
to the IVRE sub-engine of the controller doing the routing.
Each supported ESB entry is associated with either a single or a
even/odd pair of pages which provides commands to manage the source:
to EOI, to turn off the source for instance.
On a sPAPR machine, the O/S will obtain the page address of the ESB
entry associated with a source and its characteristic using the
H_INT_GET_SOURCE_INFO hcall. On PowerNV, a similar OPAL call is used.
The xive_source_notify() routine is in charge forwarding the source
event notification to the routing engine. It will be filled later on.
Greg Kurz [Tue, 27 Nov 2018 13:06:31 +0000 (14:06 +0100)]
e500: simplify IRQ wiring
The OpenPIC have 5 outputs per connected CPU. The machine init code hence
needs a bi-dimensional array (smp_cpu lines, 5 columns) to wire up the irqs
between the PIC and the CPUs.
The current code first allocates an array of smp_cpus pointers to qemu_irq
type, then it allocates another array of smp_cpus * 5 qemu_irq and fills the
first array with pointers to each line of the second array. This is rather
convoluted.
Simplify the logic by introducing a structured type that describes all the
OpenPIC outputs for a single CPU, ie, fixed size of 5 qemu_irq, and only
allocate a smp_cpu sized array of those.
This also allows to use g_new(T, n) instead of g_malloc(sizeof(T) * n)
as recommended in HACKING.
Greg Kurz [Tue, 27 Nov 2018 13:06:22 +0000 (14:06 +0100)]
mac_newworld: simplify IRQ wiring
The OpenPIC have 5 outputs per connected CPU. The machine init code hence
needs a bi-dimensional array (smp_cpu lines, 5 columns) to wire up the irqs
between the PIC and the CPUs.
The current code first allocates an array of smp_cpus pointers to qemu_irq
type, then it allocates another array of smp_cpus * 5 qemu_irq and fills the
first array with pointers to each line of the second array. This is rather
convoluted.
Simplify the logic by introducing a structured type that describes all the
OpenPIC outputs for a single CPU, ie, fixed size of 5 qemu_irq, and only
allocate a smp_cpu sized array of those.
This also allows to use g_new(T, n) instead of g_malloc(sizeof(T) * n)
as recommended in HACKING.
Implement the addex instruction introduced in ISA V3.00 in qemu tcg.
The add extended using alternate carry bit (addex) instruction performs
the same operation as the add extended (adde) instruction, but using the
overflow (ov) field in the fixed point exception register (xer) as the
carry in and out instead of the carry (ca) field.
The instruction has a Z23-form, not an XO form, as follows:
However since the only valid form of the instruction defined so far is
CY = 0, we can treat this like an XO form instruction.
There is no dot form (addex.) of the instruction and the summary overflow
(so) bit in the xer is not modified by this instruction.
For simplicity we reuse the gen_op_arith_add function and add a function
argument to specify where the carry in input should come from and the
carry out output be stored (note must be the same location).
Serhii Popovych [Thu, 22 Nov 2018 13:19:27 +0000 (08:19 -0500)]
spapr: Fix ibm,max-associativity-domains property number of nodes
Laurent Vivier reported off by one with maximum number of NUMA nodes
provided by qemu-kvm being less by one than required according to
description of "ibm,max-associativity-domains" property in LoPAPR.
It appears that I incorrectly treated LoPAPR description of this
property assuming it provides last valid domain (NUMA node here)
instead of maximum number of domains.
After fix applied numactl(8) reports 3 nodes available and memory
plugged into node 2 as expected.
From David Gibson:
------------------
Qemu makes a distinction between "non NUMA" (nb_numa_nodes == 0) and
"NUMA with one node" (nb_numa_nodes == 1). But from a PAPR guests's
point of view these are equivalent. I don't want to present two
different cases to the guest when we don't need to, so even though the
guest can handle it, I'd prefer we put a '1' here for both the
nb_numa_nodes == 0 and nb_numa_nodes == 1 case.
This consolidates everything discussed previously on mailing list.
The (only) obvious use for these macros is constructing and parsing guest
visible register fields. But the way they're constructed, they're only
valid when used on a *host* long, whose size shouldn't be visible to the
guest at all.
They also have no current users, so just get rid of them.
Peter Xu [Thu, 20 Dec 2018 05:40:37 +0000 (13:40 +0800)]
x86-iommu: turn on IR by default if proper
When the user didn't specify "intremap" for the IOMMU device, we turn
it on by default if it is supported. This will turn IR on for the
default Q35 platform as long as the IOMMU device is specified on new
kernels.
Peter Xu [Thu, 20 Dec 2018 05:40:36 +0000 (13:40 +0800)]
x86-iommu: switch intr_supported to OnOffAuto type
Switch the intr_supported variable from a boolean to OnOffAuto type so
that we can know whether the user specified it or not. With that
we'll have a chance to help the user to choose more wisely where
possible. Introduce x86_iommu_ir_supported() to mask these changes.
Peter Xu [Thu, 20 Dec 2018 05:40:35 +0000 (13:40 +0800)]
q35: set split kernel irqchip as default
Starting from QEMU 4.0, let's specify "split" as the default value for
kernel-irqchip.
So for QEMU>=4.0 we'll have: allowed=Y,required=N,split=Y
for QEMU<=3.1 we'll have: allowed=Y,required=N,split=N
(omitting all the "kernel_irqchip_" prefix)
Note that this will let the default q35 machine type to depend on
Linux version 4.4 or newer because that's where split irqchip is
introduced in kernel. But it's fine since we're boosting supported
Linux version for QEMU 4.0 to around Linux 4.5. For more information
please refer to the discussion on AMD's RDTSCP:
Alex Williamson [Mon, 18 Jan 2016 23:06:03 +0000 (16:06 -0700)]
pci: Adjust PCI config limit based on bus topology
A conventional PCI bus does not support config space accesses above
the standard 256 byte configuration space. PCIe-to-PCI bridges are
not permitted to forward transactions if the extended register address
field is non-zero and must handle it as an unsupported request (PCIe
bridge spec rev 1.0, 4.1.3, 4.1.4). Therefore, we should not support
extended config space if there is a conventional bus anywhere on the
path to a device.
This is a preparation for multi-stage hotplug handlers, whereby the bus
hotplug handler is overwritten by the machine hotplug handler. This handler
will then pass control to the bus hotplug handler. So to get this running
cleanly, we also have to make sure to go via the hotplug handler chain when
actually unplugging a device after an unplug request. Lookup the hotplug
handler and call "unplug".
This is a preparation for multi-stage hotplug handlers, whereby the bus
hotplug handler is overwritten by the machine hotplug handler. This handler
will then pass control to the bus hotplug handler. So to get this running
cleanly, we also have to make sure to go via the hotplug handler chain when
actually unplugging a device after an unplug request. Lookup the hotplug
handler and call "unplug".
pci: Reuse pci-bridge hotplug handler handlers for pcie-pci-bridge
These functions are essentially the same, we only have to use
object_get_typename() for reporting errors. So let's share the
implementation of hotplug handler callbacks.
This is a preparation for multi-stage hotplug handlers, whereby the bus
hotplug handler is overwritten by the machine hotplug handler. This handler
will then pass control to the bus hotplug handler. So to get this running
cleanly, we also have to make sure to go via the hotplug handler chain when
actually unplugging a device after an unplug request. Lookup the hotplug
handler and call "unplug".
projects / qemu.git / log