At the moment VFIO PCI device initialization works as follows:
vfio_realize
vfio_get_group
vfio_connect_container
register memory listeners (1)
update QEMU groups lists
vfio_kvm_device_add_group
Then (example for pseries) the machine reset hook triggers region_add()
for all regions where listeners from (1) are listening:
This scheme works fine until we need to handle VFIO PCI device hotplug
and we want to enable PPC64/sPAPR in-kernel TCE acceleration on,
i.e. after PCI hotplug we need a place to call
ioctl(vfio_kvm_device_fd, KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE).
Since the ioctl needs a LIOBN fd (from sPAPRTCETable) and a IOMMU group fd
(from VFIOGroup), vfio_listener_region_add() seems to be the only place
for this ioctl().
However this only works during boot time because the machine reset
happens strictly after all devices are finalized. When hotplug happens,
vfio_listener_region_add() is called when a memory listener is registered
but when this happens:
1. new group is not added to the container->group_list yet;
2. VFIO KVM device is unaware of the new IOMMU group.
This moves bits around to have all necessary VFIO infrastructure
in place for both initial startup and hotplug cases.
[aw: ie, register vfio groups with kvm prior to memory listener
registration such that kvm-vfio pseudo device ioctls are available
during the region_add callback]
* remotes/kraxel/tags/audio-20170717-pull-request:
audio/adlib: remove limitation of one adlib card
audio/fmopl: modify timer callback to give opaque and channel parameters in two arguments
audio: st_rate_flow exist a infinite loop
Alexander Graf [Wed, 12 Jul 2017 12:34:28 +0000 (14:34 +0200)]
PPC: E500: Update u-boot to v2017.07
Quite a while has passed since we last updated U-Boot for e500. This patch
bumps it to the last released version 2017.07 to make sure users don't feel
like they're using out of date software.
Peter Maydell [Mon, 17 Jul 2017 11:52:59 +0000 (12:52 +0100)]
Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-2.10-20170717' into staging
ppc patch queue 2017-07-17
This pull requests supersedes the one from 2017-07-14. That one had a
couple of subtle regressions: there was a build error for mingw32, and
an instance_size which was theoretically wrong everywhere, but only
actually bit on the Travis OSX build.
There are two major batches in this set, rather than the usual
collection of assorted fixes.
* More DRC cleanup. This gets the state management into a state
which should fix many of the hotplug+migration problems we've
had. Plus it gets the migration stream format into something
well defined and pretty minimal which we can reasonably support
into the future.
* Hashed Page Table resizing. It's been a while since this was
posted, but it's been through several previous rounds of review.
The kernel parts (both guest and host) are merged in 4.11, so
this is the only remaining piece left to allow resizing of the
HPT in a running guest.
Peter Maydell [Mon, 17 Jul 2017 10:46:36 +0000 (11:46 +0100)]
Merge remote-tracking branch 'remotes/famz/tags/block-and-testing-pull-request' into staging
# gpg: Signature made Mon 17 Jul 2017 04:47:05 BST
# gpg: using RSA key 0xCA35624C6A9171C6
# gpg: Good signature from "Fam Zheng <[email protected]>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6
* remotes/famz/tags/block-and-testing-pull-request:
travis: add no-TCG build
docker.py: Improve subprocess exit code handling
docker.py: Drop infile parameter
docker: Don't enable networking as a side-effect of DEBUG=1
ssh: support I/O from any AioContext
sheepdog: add queue_lock
qed: protect table cache with CoMutex
qed: introduce bdrv_qed_init_state
block: invoke .bdrv_drain callback in coroutine context and from AioContext
qed: move tail of qed_aio_write_main to qed_aio_write_{cow, alloc}
vvfat: make it thread-safe
vpc: make it thread-safe
vdi: make it thread-safe
coroutine-lock: add qemu_co_rwlock_downgrade and qemu_co_rwlock_upgrade
qcow2: call CoQueue APIs under CoMutex
Alexander Graf [Wed, 12 Jul 2017 12:43:45 +0000 (14:43 +0200)]
vnc: Set default kbd delay to 10ms
The current VNC default keyboard delay is 1ms. With that we're constantly
typing faster than the guest receives keyboard events from an XHCI attached
USB HID device.
The default keyboard delay time in the input layer however is 10ms. I don't know
how that number came to be, but empirical tests on some OpenQA driven ARM
systems show that 10ms really is a reasonable default number for the delay.
This patch moves the VNC delay also to 10ms. That way our default is much
safer (good!) and also consistent with the input layer default (also good!).
If a voice recording equipment is opened for a long time(several days)
in windows guest, rate->ipos will overflow and rate->opos will never
have a chance to change. It will result to a infinite loop.
Peter Maydell [Mon, 17 Jul 2017 09:02:22 +0000 (10:02 +0100)]
Merge remote-tracking branch 'remotes/thibault/tags/samuel-thibault' into staging
slirp updates
# gpg: Signature made Sat 15 Jul 2017 13:30:03 BST
# gpg: using RSA key 0xB0A51BF58C9179C5
# gpg: Good signature from "Samuel Thibault <[email protected]>"
# gpg: aka "Samuel Thibault <[email protected]>"
# gpg: aka "Samuel Thibault <[email protected]>"
# gpg: aka "Samuel Thibault <[email protected]>"
# gpg: aka "Samuel Thibault <[email protected]>"
# gpg: aka "Samuel Thibault <[email protected]>"
# gpg: aka "Samuel Thibault <[email protected]>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 900C B024 B679 31D4 0F82 304B D017 8C76 7D06 9EE6
# Subkey fingerprint: AEBF 7448 FAB9 453A 4552 390E B0A5 1BF5 8C91 79C5
* remotes/thibault/tags/samuel-thibault:
slirp: Handle error returns from sosendoob()
slirp: Handle error returns from slirp_send() in sosendoob()
slirp: fork_exec(): Don't close() a negative number in fork_exec()
slirp: use DIV_ROUND_UP
Christian Nilsson (1):
[intel] Add INTEL_NO_PHY_RST for I219-LM (2)
David Decotigny (2):
[build] Return const char * from uuid_ntoa()
[af_packet] Add new AF_PACKET driver for Linux
Jason Wang (1):
[virtio] Support VIRTIO_NET_F_IOMMU_PLATFORM
Jerone Young (1):
[intel] Add support for I219-V in 7th Gen Intel NUC
Konrad Adamczyk (1):
[thunderx] Don't disable NIC when exiting from iPXE
Ladi Prosek (3):
[virtio] Cap queue size to MAX_QUEUE_NUM
[virtio] Simplify virtqueue shutdown
[virtio] Remove queue size limit in legacy virtio
Martin Habets (1):
[sfc] Add driver for Solarflare SFC8XXX adapters
Michael Brown (159):
[interface] Provide intf_reinit() to reinitialise nullified interfaces
[iscsi] Avoid potential infinite loops during shutdown
[efi] Add basic EFI SAN booting capability
[undi] Allocate base memory before calling UNDI loader entry point
[romprefix] Avoid using PMM-allocated memory in UNDI loader entry point
[undi] Clean up driver and device name information
[prefix] Remove impossible progress message
[prefix] Include diagnostic information within progress messages
[undi] Try matching UNDI ROMs in BIOS enumeration order
[efi] Work around temporal anomaly encountered during ExitBootServices()
[ipv4] Accept unicast packets for the local network broadcast address
[build] Add %.vhd target for building VM bootable disk images
[virtio] Use separate RX and TX empty header buffers
[cloud] Add ability to retrieve Google Compute Engine metadata
[virtio] Use host-specified MTU when available
[netdevice] Allow MTU to be changed at runtime
[cloud] Show CPU vendor and model in example cloud boot scripts
[hyperv] Ignore unsolicited VMBus messages
[pic8259] Fix definitions for "read IRR" and "read ISR" commands
[efi] Fix building elf2efi.c when -fpic is enabled by default
[interface] Avoid unnecessary reference counting in intf_unplug()
[interface] Remove misleading comment
[interface] Unplug interface before calling intf_close() in intf_shutdown()
[netdevice] Limit MTU by hardware maximum frame length
[cpuid] Provide cpuid_supported() to test for supported functions
[time] Allow timer to be selected at runtime
[hyperv] Provide timer based on the 10MHz time reference count MSR
[int13] Avoid potential division by zero
[int13] Test correct return status from INT 13 calls
[settings] Add "unixtime" builtin setting to expose the current time
[time] Report attempts to use timers before initialisation
[interface] Provide the ability to shut down multiple interfaces
[http] Cleanly shut down potentially looped interfaces
[efi] Add missing SANBOOT_PROTO_HTTP to EFI default configuration
[block] Remove spurious comments
[block] Centralise SAN device abstraction
[block] Centralise "san-drive" setting
[int13] Refactor to use centralised SAN device abstraction
[efi] Refactor to use centralised SAN device abstraction
[block] Retry any SAN device operation
[iscsi] Use intfs_shutdown() when shutting down multiple interfaces
[scsi] Use intfs_shutdown() when shutting down multiple interfaces
[block] Use intfs_shutdown() when shutting down multiple interfaces
[scsi] Avoid duplicate calls to scsicmd_close()
[build] Provide common ARRAY_SIZE() definition
[efi] Update to current EDK2 headers
[efi] Add EFI_ACPI_TABLE_PROTOCOL header and GUID definition
[efi] Provide ACPI table description for SAN devices
[efi] Skip cable detection at initialisation where possible
[undi] Move PXE API caller back into UNDI driver
[dhcp] Allow vendor class to be changed in DHCP requests
[hermon] Avoid potential integer overflow when calculating memory mappings
[arbel] Avoid potential integer overflow when calculating memory mappings
[xfer] Ensure va_end() is called on failure path
[nfs] Fix double free bug on error path
[linda] Use correct length for memset()
[qib7322] Use correct length for memset()
[sis900] Remove extraneous memset() with incorrect length
[802.11] Remove redundant NULL pointer check after dereference
[crypto] Free correct pointer on the error path
[librm] Fail gracefully if asked to ioremap() a zero length
[usb] Use correct length for memcpy()
[mucurses] Attempt to fix test for empty string
[mucurses] Attempt to fix keypress processing logic
[mucurses] Attempt to fix resource leaks
[hyperv] Fix resource leaks on error path
[slam] Fix resource leak on error path
[slam] Avoid NULL pointer dereference in slam_pull_value()
[eoib] Avoid passing a NULL I/O buffer to netdev_tx_complete_err()
[http] Add missing check for memory allocation failure
[mucurses] Attempt to fix use of uninitialised buffer with strcat()
[xhci] Avoid accessing beyond end of endpoint context array
[build] Avoid confusing sparse in single-argument DBG() macros
[infiniband] Return status code from ib_create_cq() and ib_create_qp()
[infiniband] Return status code from ib_create_mi()
[block] Quell spurious Coverity size mismatch warning
[ath] Add missing break statements
[pixbuf] Avoid potential division by zero
[usb] Use correct length for memcpy()
[xen] Use standard calling pattern for asprintf()
[tcp] Use correct length for memset()
[video_subr] Use memmove() for overlapping memory copy
[arbel] Assert that mapping length is non-zero
[hermon] Assert that mapping length is non-zero
[tlan] Guard against failure to identify chip
[w89c840] Avoid potential array overrun
[sis190] Avoid NULL pointer dereference
[mucurses] Ensure SLK labels are always terminated
[coverity] Add Coverity user model
[malloc] Track maximum heap usage
[travis] Add minimal .travis.yml file
[travis] Build and run the unit test suite
[travis] Integrate with Coverity Scan
[rtl818x] Fix resource leak on error path
[pcnet32] Eliminate redundant register read
[iobuf] Increase minimum I/O buffer size to 128 bytes
[vxge] Fix use of stale I/O buffer on error path
[scsi] Avoid duplicate call to scsicmd_close() on TEST UNIT READY failure
[block] Add dummy SAN device
[block] Add basic multipath support
[int13] Improve geometry guessing for unaligned partitions
[int13con] Avoid overwriting random portions of SAN boot disks
[time] Add sleep_fixed() function to sleep without checking for Ctrl-C
[block] Allow SAN retry count to be reconfigured
[block] Add a small delay between attempts to reopen SAN targets
[block] Retry reopening indefinitely for multipath devices
[block] Gracefully close SAN device if registration fails
[linux] Use dummy SAN device
[block] Ignore redundant xfer_window_changed() messages
[block] Describe all SAN devices via ACPI tables
[iscsi] Do not install iBFT when no iSCSI targets exist
[http] Notify data transfer interface when underlying connection is ready
[mucurses] Fix erroneous __nonnull attribute
[build] Avoid implicit-fallthrough warnings on GCC 7
[linux] Fix building with kernel 4.11 headers
[scsi] Retry TEST UNIT READY command
[libc] Add stdbool.h standard header
[efi] Fix typo in efi_acpi_table_protocol_guid
[efi] Add efi_sprintf() and efi_vsprintf()
[block] Allow use of a non-default EFI SAN boot filename
[intel] Show original CTRL and STATUS values in debugging output
[intel] Do not enable ASDE on i350 backplane NIC
[block] Provide sandev_read() and sandev_write() as global symbols
[block] Provide abstraction to allow system to be quiesced
[hyperv] Do not fail if guest OS ID MSR is already set
[hyperv] Remove redundant return status code from mapping functions
[hyperv] Cope with Windows Server 2016 enlightenments
[efi] Standardise PCI debug messages
[iscsi] Always send FirstBurstLength parameter
[iscsi] Fix iBFT when no explicit initiator name setting exists
[xen] Provide 18 4kB receive buffers to work around xen-netback bug
[efi] Prevent EFI code from being linked in to non-EFI builds
[tls] Keep cipherstream window open until TLS negotiation is complete
[settings] Extend numerical setting tags to 64 bits
[acpi] Make acpi_find_rsdt() a per-platform method
[efi] Provide access to ACPI tables
[acpi] Expose ACPI tables via settings mechanism
[syslog] Handle backspace characters
[hdprefix] Avoid attempts to read beyond the end of the disk
[usb] Allow for USB network devices with no interrupt endpoint
[build] Use -no-pie on newer versions of gcc
[ecm] Display invalid MAC address strings in debug messages
[cpuid] Allow input %ecx value to be specified
[crypto] Expose RSA_CTX_SIZE constant
[crypto] Expose asn1_grow()
[crypto] Provide asn1_built() to construct a cursor from a builder
[crypto] Expose pem_asn1() for use with non-image data
[exanic] Add driver for Exablaze ExaNIC cards
[usb] Use non-zero language ID to retrieve strings
[mucurses] Avoid potential division by zero
[tls] Support RFC5746 secure renegotiation
[smscusb] Abstract out common SMSC USB device functionality
[smsc95xx] Use common SMSC USB device functionality
[smsc75xx] Use common SMSC USB device functionality
[smscusb] Add ability to read MAC address from OTP
[smscusb] Move non-inline register access functions to smscusb.c
[smscusb] Allow for alternative PHY register layouts
[smsc75xx] Expose functionality shared with LAN78xx devices
[lan78xx] Add driver for Microchip LAN78xx USB Ethernet NICs
Mika Tiainen (1):
[intel] Add INTEL_NO_PHY_RST for I219-V
Mike McCormack (1):
[sky2] Use 32-bit read to read Y2_VAUX_AVAIL
Raed Salem (2):
[golan] Update Connect-IB, ConnectX-4 and ConnectX-4 Lx (Infiniband) support
[golan] Bug fixes and improved paging allocation method
Vishvananda Ishaya (1):
[intel] Reset all virtual function settings
Vishvananda Ishaya Abrams (1):
[iscsi] Don't close when receiving NOP-In
target/ppc: fix CPU hotplug when radix is enabled (TCG)
But when a guest initializes radix mode, it issues a H_REGISTER_PROC_TBL
to update the LPCR of all CPUs. Hot-plugged CPUs inherit from the same
setting under KVM but not under TCG. So, Let's check for radix and update
the default LPCR to keep new CPUs in sync.
David Gibson [Wed, 12 Jul 2017 07:56:55 +0000 (17:56 +1000)]
pseries: Allow HPT resizing with KVM
So far, qemu implements the PAPR Hash Page Table (HPT) resizing extension
with TCG. The same implementation will work with KVM PR, but we don't
currently allow that. For KVM HV we can only implement resizing with the
assistance of the host kernel, which needs a new capability and ioctl()s.
This patch adds support for testing the new KVM capability and implementing
the resize in terms of KVM facilities when necessary. If we're running on
a kernel which doesn't have the new capability flag at all, we fall back to
testing for PR vs. HV KVM using the same hack that we already use in a
number of places for older kernels.
David Gibson [Wed, 12 Jul 2017 07:56:06 +0000 (17:56 +1000)]
pseries: Use smaller default hash page tables when guest can resize
We've now implemented a PAPR extension allowing PAPR guest to resize
their hash page table (HPT) during runtime.
This patch makes use of that facility to allocate smaller HPTs by default.
Specifically when a guest is aware of the HPT resize facility, qemu sizes
the HPT to the initial memory size, rather than the maximum memory size on
the assumption that the guest will resize its HPT if necessary for hot
plugged memory.
When the initial memory size is much smaller than the maximum memory size
(a common configuration with e.g. oVirt / RHEV) then this can save
significant memory on the HPT.
If the guest does *not* advertise HPT resize awareness when it makes the
ibm,client-architecture-support call, qemu resizes the HPT for maxmimum
memory size (unless it's been configured not to allow such guests at all).
For now we make that reallocation assuming the guest has not yet used the
HPT at all. That's true in practice, but not, strictly, an architectural
or PAPR requirement. If we need to in future we can fix this by having
the client-architecture-support call reboot the guest with the revised
HPT size (the client-architecture-support call is explicitly permitted to
trigger a reboot in this way).
David Gibson [Wed, 12 Jul 2017 07:53:17 +0000 (17:53 +1000)]
pseries: Enable HPT resizing for 2.10
We've now implemented a PAPR extensions which allows PAPR guests (i.e.
"pseries" machine type) to resize their hash page table during runtime.
However, that extension is only enabled if explicitly chosen on the
command line. This patch enables it by default for spapr-2.10, but leaves
it disabled (by default) for older machine types.
David Gibson [Fri, 12 May 2017 05:46:49 +0000 (15:46 +1000)]
pseries: Implement HPT resizing
This patch implements hypercalls allowing a PAPR guest to resize its own
hash page table. This will eventually allow for more flexible memory
hotplug.
The implementation is partially asynchronous, handled in a special thread
running the hpt_prepare_thread() function. The state of a pending resize
is stored in SPAPR_MACHINE->pending_hpt.
The H_RESIZE_HPT_PREPARE hypercall will kick off creation of a new HPT, or,
if one is already in progress, monitor it for completion. If there is an
existing HPT resize in progress that doesn't match the size specified in
the call, it will cancel it, replacing it with a new one matching the
given size.
The H_RESIZE_HPT_COMMIT completes transition to a resized HPT, and can only
be called successfully once H_RESIZE_HPT_PREPARE has successfully
completed initialization of a new HPT. The guest must ensure that there
are no concurrent accesses to the existing HPT while this is called (this
effectively means stop_machine() for Linux guests).
For now H_RESIZE_HPT_COMMIT goes through the whole old HPT, rehashing each
HPTE into the new HPT. This can have quite high latency, but it seems to
be of the order of typical migration downtime latencies for HPTs of size
up to ~2GiB (which would be used in a 256GiB guest).
In future we probably want to move more of the rehashing to the "prepare"
phase, by having H_ENTER and other hcalls update both current and
pending HPTs. That's a project for another day, but should be possible
without any changes to the guest interface.
David Gibson [Fri, 12 May 2017 05:46:11 +0000 (15:46 +1000)]
pseries: Stubs for HPT resizing
This introduces stub implementations of the H_RESIZE_HPT_PREPARE and
H_RESIZE_HPT_COMMIT hypercalls which we hope to add in a PAPR
extension to allow run time resizing of a guest's hash page table. It
also adds a new machine property for controlling whether this new
facility is available.
For now we only allow resizing with TCG, allowing it with KVM will require
kernel changes as well.
Finally, it adds a new string to the hypertas property in the device
tree, advertising to the guest the availability of the HPT resizing
hypercalls. This is a tentative suggested value, and would need to be
standardized by PAPR before being merged.
Greg Kurz [Wed, 12 Jul 2017 09:48:39 +0000 (11:48 +0200)]
spapr: fix potential memory leak in spapr_core_plug()
Since commit 5c1da81215c7 ("spapr: Remove unnecessary differences between
hotplug and coldplug paths"), the CPU DT for the DRC is always allocated.
This causes a memory leak for pseries-2.6 and older machine types, that
don't support CPU hotplug and don't allocate DRCs for CPUs.
David Gibson [Thu, 8 Jun 2017 13:55:03 +0000 (23:55 +1000)]
spapr: Implement DR-indicator for physical DRCs only
According to PAPR, the DR-indicator should only be valid for physical DRCs,
not logical DRCs. At the moment we implement it for all DRCs, so restrict
it to physical ones only.
We move the state to the physical DRC subclass, which means adding some
QOM boilerplate to handle the newly distinct type.
Most of the time, the state of a DRC object is contained in the single
'state' variable. However, during the transition from UNISOLATE to
CONFIGURED state requires multiple calls to the ibm,configure-connector
RTAS call to retrieve the device tree for the attached device. We need
some extra state to keep track of where we're up to in delivering the
device tree information to the guest.
Currently that extra state is in a sPAPRConfigureConnectorState
substructure which is only allocated when we're in the middle of the
configure connector process. That sounds like a good idea, but the extra
state is only two integers - on many platforms that will take up the same
room as the (maybe NULL) ccs pointer even before malloc() overhead. Plus
it's another object whose lifetime we need to manage. In short, it's not
worth it.
So, fold the sPAPRConfigureConnectorState substructure directly into the
DRC object.
Previously the structure was allocated lazily when the configure-connector
call discovers it's not there. Now, we need to initialize the subfields
pre-emptively, as soon as we enter UNISOLATE state.
Although it's not strictly necessary (the field values should only ever
be consulted when in UNISOLATE state), we try to keep them at -1 when in
other states, as a debugging aid.
David Gibson [Tue, 20 Jun 2017 13:57:48 +0000 (21:57 +0800)]
spapr: Consolidate DRC state variables
Each DRC has three fields describing its state: isolation_state,
allocation_state and configured. At first this seems like a reasonable
representation, since its based directly on the PAPR defined
isolation-state and allocation-state indicators. However:
* Only a few combinations of the two fields' values are permitted
* allocation_state isn't used at all for physical DRCs
* The indicators are write only so they don't really have a well
defined current value independent of each other
This replaces these variables with a single state variable, whose names
and numbers are based on the diagram in LoPAPR section 13.4. Along with
this we add code to check the current state on various operations and make
sure the requested transition is permitted.
Strictly speaking, this makes guest visible changes to behaviour (since we
probably allowed some transitions we shouldn't have before). However, a
hypothetical guest broken by that wasn't PAPR compliant, and probably
wouldn't have worked under PowerVM.
David Gibson [Tue, 20 Jun 2017 13:02:41 +0000 (21:02 +0800)]
spapr: Cleanups relating to DRC awaiting_release field
'awaiting_release' indicates that the host has requested an unplug of the
device attached to the DRC, but the guest has not (yet) put the device
into a state where it is safe to complete removal.
1. Rename it to 'unplug_requested' which to me at least is clearer
2. Remove the ->release_pending() method used to check this from outside
spapr_drc.c. The method only plausibly has one implementation, so use
a plain function (spapr_drc_unplug_requested()) instead.
3. Remove it from the migration stream. Attempting to migrate mid-unplug
is broken not just for spapr - in general management has no good way to
determine if the device should be present on the destination or not. So,
until that's fixed, there's no point adding extra things to the stream.
David Gibson [Tue, 4 Jul 2017 11:07:14 +0000 (21:07 +1000)]
spapr: Refactor spapr_drc_detach()
This function has two unused parameters - remove them.
It also sets awaiting_release on all paths, except one. On that path
setting it is harmless, since it will be immediately cleared by
spapr_drc_release(). So factor it out of the if statements.
David Gibson [Thu, 13 Jul 2017 00:52:39 +0000 (10:52 +1000)]
spapr: Abort on delete failure in spapr_drc_release()
We currently ignore errors from the object_property_del() in
spapr_drc_release(). But the only way that could fail is if the property
doesn't exist, in which case it's a bug that we're in spapr_drc_release()
at all. So change from ignoring to abort()ing on errors.
David Gibson [Thu, 13 Jul 2017 00:45:35 +0000 (10:45 +1000)]
spapr: Simplify unplug path
spapr_lmb_release() and spapr_core_release() call hotplug_handler_unplug()
which after a bunch of indirection calls spapr_memory_unplug() or
spapr_core_unplug(). But we already know which is the appropriate thing
to call here, so we can just fold it directly into the release function.
Once that's done, there's no need for an hc->unplug method in the spapr
machine at all: since we also have an hc->unplug_request method, the
hotplug core will never use ->unplug.
David Gibson [Mon, 3 Jul 2017 10:20:53 +0000 (20:20 +1000)]
spapr: Remove 'awaiting_allocation' DRC flag
The awaiting_allocation flag in the DRC was introduced by aab9913
"spapr_drc: Prevent detach racing against attach for CPU DR", allegedly to
prevent a guest crash on racing attach and detach. Except.. information
from the BZ actually suggests a qemu crash, not a guest crash. And there
shouldn't be a problem here anyway: if the guest has already moved the DRC
away from UNUSABLE state, the detach would already be deferred, and if it
hadn't it should be safe to detach it (the guest should fail gracefully
when it attempts to change the allocation state).
I think this was probably just a bandaid for some other problem in the
state management. So, remove awaiting_allocation and associated code.
Laurent Vivier [Fri, 9 Jun 2017 11:08:10 +0000 (13:08 +0200)]
spapr: Treat devices added before inbound migration as coldplugged
When migrating a guest which has already had devices hotplugged,
libvirt typically starts the destination qemu with -incoming defer,
adds those hotplugged devices with qmp, then initiates the incoming
migration.
This causes problems for the management of spapr DRC state. Because
the device is treated as hotplugged, it goes into a DRC state for a
device immediately after it's plugged, but before the guest has
acknowledged its presence. However, chances are the guest on the
source machine *has* acknowledged the device's presence and configured
it.
If the source has fully configured the device, then DRC state won't be
sent in the migration stream: for maximum migration compatibility with
earlier versions we don't migrate DRCs in coldplug-equivalent state.
That means that the DRC effectively changes state over the migrate,
causing problems later on.
In addition, logging hotplug events for these devices isn't what we
want because a) those events should already have been issued on the
source host and b) the event queue should get wiped out by the
incoming state anyway.
In short, what we really want is to treat devices added before an
incoming migration as if they were coldplugged.
To do this, we first add a spapr_drc_hotplugged() helper which
determines if the device is hotplugged in the sense relevant for DRC
state management. We only send hotplug events when this is true.
Second, when we add a device which isn't hotplugged in this sense, we
force a reset of the DRC state - this ensures the DRC is in a
coldplug-equivalent state (there isn't usually a system reset between
these device adds and the incoming migration).
This is based on an earlier patch by Laurent Vivier, cleaned up and
extended.
David Gibson [Wed, 12 Jul 2017 01:55:53 +0000 (11:55 +1000)]
spapr: Minor cleanups to events handling
The rtas_error_log structure is marked packed, which strongly suggests its
precise layout is important to match an external interface. Along with
that one could expect it to have a fixed endianness to match the same
interface. That used to be the case - matching the layout of PAPR RTAS
event format and requiring BE fields.
Now, however, it's only used embedded within sPAPREventLogEntry with the
fields in native order, since they're processed internally.
Clear that up by removing the nested structure in sPAPREventLogEntry.
struct rtas_error_log is moved back to spapr_events.c where it is used as
a temporary to help convert the fields in sPAPREventLogEntry to the correct
in memory format when delivering an event to the guest.
In racing situations between hotplug events and migration operation,
a rtas hotplug event could have not yet be delivered to the source
guest when migration is started. In this case the pending_events of
spapr state need be transmitted to the target so that the hotplug
event can be finished on the target.
To achieve the minimal VMSD possible to migrate the pending_events list,
this patch makes the changes in spapr_events.c:
- 'log_type' of sPAPREventLogEntry struct deleted. This information can be
derived by inspecting the rtas_error_log summary field. A new function
called 'spapr_event_log_entry_type' was added to retrieve the type of
a given sPAPREventLogEntry.
- sPAPREventLogEntry, epow_log_full and hp_log_full were redesigned. The
only data we're going to migrate in the VMSD is the event log data itself,
which can be divided in two parts: a rtas_error_log header and an extended
event log field. The rtas_error_log header contains information about the
size of the extended log field, which can be used inside VMSD as the size
parameter of the VBUFFER_ALOC field that will store it. To allow this use,
the header.extended_length field must be exposed inline to the VMSD instead
of embedded into a 'data' field that holds everything. With this in mind,
the following changes were done:
* a new 'header' field was added to sPAPREventLogEntry. This field holds a
a struct rtas_error_log inline.
* the declaration of the 'rtas_error_log' struct was moved to spapr.h
to be visible to the VMSD macros.
* 'data' field of sPAPREventLogEntry was renamed to 'extended_log' and
now holds only the contents of the extended event log.
* 'struct rtas_error_log hdr' were taken away from both epow_log_full
and hp_log_full. This information is now available at the header field of
sPAPREventLogEntry.
* epow_log_full and hp_log_full were renamed to epow_extended_log and
hp_extended_log respectively. This rename makes it clearer to understand
the new purpose of both structures: hold the information of an extended
event log field.
* spapr_powerdown_req and spapr_hotplug_req_event now creates a
sPAPREventLogEntry structure that contains the full rtas log entry.
* rtas_event_log_queue and rtas_event_log_dequeue now receives a
sPAPREventLogEntry pointer as a parameter instead of a void pointer.
- the endianess of the sPAPREventLogEntry header is now native instead
of be32. We can use the fields in native endianess internally and write
them in be32 in the guest physical memory inside 'check_exception'. This
allows the VMSD inside spapr.c to read the correct size of the
entended_log field.
- inside spapr.c, pending_events is put in a subsection in the spapr state
VMSD to make sure migration across different versions is not broken.
A small change in rtas_event_log_queue and rtas_event_log_dequeue were also
made: instead of calling qdev_get_machine(), both functions now receive
a pointer to the sPAPRMachineState. This pointer is already available in
the callers of these functions and we don't need to waste resources
calling qdev() again.
David Gibson [Mon, 17 Jul 2017 04:51:09 +0000 (14:51 +1000)]
spapr: Remove unnecessary instance_size specifications from DRC subtypes
All the DRC subtypes explicitly list instance_size in TypeInfo (all as
sizeof(sPAPRDRConnector). This isn't necessary, since if it's not listed
it will be derived from the parent type.
Worse, this is dangerous, because if a subtype is changed in future to
have a larger structure, then subtypes of that subtype also need to have
instance_size changed, or it will lead to hard to track memory corruption
bugs.
docker: Don't enable networking as a side-effect of DEBUG=1
When trying to debug problems with tests it is natural to set
DEBUG=1 when starting the docker environment. Unfortunately
this has a side-effect of enabling an eth0 network interface
in the container, which changes the operating environment of
the test suite. IOW tests with fail may suddenly start
working again if DEBUG=1 is set, due to changed network setup.
Add a separate NETWORK variable to allow enablement of
networking separately from DEBUG=1. This can be used in two
ways. To enable the default docker network backend
make docker-test-build@fedora NETWORK=1
while to enable a specific network backend, eg join the network
associated with the container 'wibble':
make docker-test-build@fedora NETWORK=container:wibble
Paolo Bonzini [Thu, 29 Jun 2017 13:27:49 +0000 (15:27 +0200)]
ssh: support I/O from any AioContext
The coroutine may run in a different AioContext, causing the
fd handler to busy wait. Fix this by resetting the handler
in restart_coroutine, before the coroutine is restarted.
Paolo Bonzini [Thu, 29 Jun 2017 13:27:46 +0000 (15:27 +0200)]
qed: introduce bdrv_qed_init_state
This will be used in the next patch, which will call bdrv_qed_do_open
with a CoMutex taken. bdrv_qed_init_state provides a nice place to
initialize it.
Paolo Bonzini [Thu, 29 Jun 2017 13:27:41 +0000 (15:27 +0200)]
vdi: make it thread-safe
The VirtualBox driver is using a mutex to order all allocating writes,
but it is not protecting accesses to the bitmap because they implicitly
happen under the AioContext mutex. Change this to use a CoRwlock
explicitly.
Paolo Bonzini [Thu, 29 Jun 2017 13:27:40 +0000 (15:27 +0200)]
coroutine-lock: add qemu_co_rwlock_downgrade and qemu_co_rwlock_upgrade
These functions are more efficient in the presence of contention.
qemu_co_rwlock_downgrade also guarantees not to block, which may
be useful in some algorithms too.
Peter Maydell [Fri, 14 Jul 2017 11:12:25 +0000 (12:12 +0100)]
slirp: Handle error returns from sosendoob()
sosendoob() can return a failure code, but all its callers ignore it.
This is OK in sbappend(), as the comment there states -- we will try
again later in sowrite(). Add a (void) cast to tell Coverity so.
In sowrite() we do need to check the return value -- we should handle
a write failure in sosendoob() the same way we handle a write failure
for the normal data.
Peter Maydell [Mon, 5 Jun 2017 16:19:35 +0000 (17:19 +0100)]
slirp: Handle error returns from slirp_send() in sosendoob()
The code in sosendoob() assumes that slirp_send() always
succeeds, but it might return an OS error code (for instance
if the other end has disconnected). Catch these and return
the caller either -1 on error or the number of urgent bytes
actually written. (None of the callers check this return
value currently, though.)
Peter Maydell [Sun, 9 Jul 2017 17:54:22 +0000 (18:54 +0100)]
slirp: fork_exec(): Don't close() a negative number in fork_exec()
In a fork_exec() error path we try to closesocket(s) when s might
be a negative number because the thing that failed was the
qemu_socket() call. Add a guard so we don't do this.
Peter Maydell [Fri, 7 Jul 2017 14:42:55 +0000 (15:42 +0100)]
hw/pci/pci.c: Use memory_region_init_rom()
Since we pass the same DeviceState object to
memory_region_init_rom_nomigrate() and vmstate_register_ram(), we can
switch to using memory_region_init_rom() instead.
(This isn't entirely obvious from the code since it is using
&pdev->qdev rather than DEVICE(pdov) for some reason, but
PCIDevice does indeed use 'qdev' for its parent DeviceState member.)
Peter Maydell [Fri, 7 Jul 2017 14:42:54 +0000 (15:42 +0100)]
hw/block/pflash_cfi01, pflash_cfi02: Use memory_region_init_rom_device()
Since we pass the same DeviceState object to
memory_region_init_rom_device_nomigrate() and vmstate_register_ram(),
we can switch to using memory_region_init_rom_device() instead.
Peter Maydell [Fri, 7 Jul 2017 14:42:53 +0000 (15:42 +0100)]
hw: Use new memory_region_init_{ram, rom, rom_device}() functions
Use the new functions memory_region_init_{ram,rom,rom_device}()
instead of manually calling the _nomigrate() version and then
vmstate_register_ram_global().
Patch automatically created using coccinelle script:
spatch --in-place -sp_file scripts/coccinelle/memory-region-init-ram.cocci -dir hw
(As it turns out, there are no instances of the rom and
rom_device functions that are caught by this script.)
Peter Maydell [Fri, 7 Jul 2017 14:42:52 +0000 (15:42 +0100)]
scripts/coccinelle/memory-region-init-ram.cocci: New script
Add a coccinelle script that can be used to automatically convert
manual sequences of
memory_region_init_ram_nomigrate()
vmstate_register_ram{,_global}()
to use the new
memory_region_init_ram()
Add new utility functions which both initialize a RAM
MemoryRegion and arrange for its contents to be migrated;
we give thes the memory_region_init_ram(), memory_region_init_rom()
and memory_region_init_rom_device() names that we just freed up
by renaming the old implementations to _nomigrate().
Peter Maydell [Fri, 7 Jul 2017 14:42:50 +0000 (15:42 +0100)]
memory: Rename memory_region_init_rom() and _rom_device() to _nomigrate()
Rename memory_region_init_rom() to memory_region_init_rom_nomigrate()
and memory_region_init_rom_device() to
memory_region_init_rom_device_nomigrate().
Peter Maydell [Fri, 7 Jul 2017 14:42:49 +0000 (15:42 +0100)]
memory: Rename memory_region_init_ram() to memory_region_init_ram_nomigrate()
Rename memory_region_init_ram() to memory_region_init_ram_nomigrate().
This leaves the way clear for us to provide a memory_region_init_ram()
which does handle migration.
Peter Maydell [Fri, 7 Jul 2017 14:42:48 +0000 (15:42 +0100)]
memory: Document that the RAM MR initializers do not handle migration
The various functions for initializing RAM MemoryRegions do not do
anything to cause the data in the MemoryRegion to be migrated.
Note in their documentation comments that this is the responsibility
of the caller.
(We will shortly add a new function that *does* do this for you.)
Peter Maydell [Fri, 14 Jul 2017 15:13:29 +0000 (16:13 +0100)]
Merge remote-tracking branch 'remotes/berrange/tags/pull-sockets-2017-07-11-3' into staging
Merge sockets 2017/07/11 v3
# gpg: Signature made Fri 14 Jul 2017 16:09:03 BST
# gpg: using RSA key 0xBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <[email protected]>"
# gpg: aka "Daniel P. Berrange <[email protected]>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF
* remotes/berrange/tags/pull-sockets-2017-07-11-3:
io: preserve ipv4/ipv6 flags when resolving InetSocketAddress
sockets: ensure we don't accept IPv4 clients when IPv4 is disabled
sockets: don't block IPv4 clients when listening on "::"
sockets: ensure we can bind to both ipv4 & ipv6 separately
io: preserve ipv4/ipv6 flags when resolving InetSocketAddress
The original InetSocketAddress struct may have has_ipv4 and
has_ipv6 fields set, which will control both the ai_family
used during DNS resolution, and later use of the V6ONLY
flag.
Currently the standalone DNS resolver code drops the
has_ipv4 & has_ipv6 flags after resolving, which means
the later bind() code won't correctly set V6ONLY.
sockets: ensure we don't accept IPv4 clients when IPv4 is disabled
Currently if you disable listening on IPv4 addresses, via the
CLI flag ipv4=off, we still mistakenly accept IPv4 clients via
the IPv6 listener socket due to IPV6_V6ONLY flag being unset.
We must ensure IPV6_V6ONLY is always set if ipv4=off
sockets: don't block IPv4 clients when listening on "::"
When inet_parse() parses the hostname, it is forcing the
has_ipv6 && ipv6 flags if the address contains a ":". This
means that if the user had set the ipv4=on flag, to try to
restrict the listener to just ipv4, an error would not have
been raised. eg
-incoming tcp:[::]:9000,ipv4
should have raised an error because listening for IPv4
on "::" is a non-sensical combination. With this removed,
we now call getaddrinfo() on "::" passing PF_INET and
so getaddrinfo reports an error about the hostname being
incompatible with the requested protocol:
qemu-system-x86_64: -incoming tcp:[::]:9000,ipv4: address resolution
failed for :::9000: Address family for hostname not supported
Likewise it is explicitly setting the has_ipv4 & ipv4
flags when the address contains only digits + '.'. This
has no ill-effect, but also has no benefit, so is removed.
sockets: ensure we can bind to both ipv4 & ipv6 separately
When binding to an IPv6 socket we currently force the
IPV6_V6ONLY flag to off. This means that the IPv6 socket
will accept both IPv4 & IPv6 sockets when QEMU is launched
with something like
-vnc :::1
While this is good for that case, it is bad for other
cases. For example if an empty hostname is given,
getaddrinfo resolves it to 2 addresses 0.0.0.0 and ::,
in that order. We will thus bind to 0.0.0.0 first, and
then fail to bind to :: on the same port. The same
problem can happen if any other hostname lookup causes
the IPv4 address to be reported before the IPv6 address.
When we get an IPv6 bind failure, we should re-try the
same port, but with IPV6_V6ONLY turned on again, to
avoid clash with any IPv4 listener.
This ensures that
-vnc :1
will bind successfully to both 0.0.0.0 and ::, and also
avoid
-vnc :1,to=2
from mistakenly using a 2nd port for the :: listener.
This is a regression due to commit 396f935 "ui: add ability to
specify multiple VNC listen addresses".
Peter Maydell [Fri, 14 Jul 2017 13:19:35 +0000 (14:19 +0100)]
Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20170714' into staging
s390x/kvm/migration/cpumodel: fixes, enhancements and cleanups
- add a network boot rom for s390 (Thomas Huth)
- migration of storage attributes like the CMMA used/unused state
- PCI related enhancements - full support for aen, ais and zpci
- migration support for css with vmstates (Halil Pasic)
- cpu model enhancements for cpu features
- guarded storage support
# gpg: Signature made Fri 14 Jul 2017 11:33:04 BST
# gpg: using RSA key 0x117BBC80B5A61C7C
# gpg: Good signature from "Christian Borntraeger (IBM) <[email protected]>"
# Primary key fingerprint: F922 9381 A334 08F9 DBAB FBCA 117B BC80 B5A6 1C7C
* remotes/borntraeger/tags/s390x-20170714: (40 commits)
s390x/gdb: add gs registers
s390x/arch_dump: also dump guarded storage control block
s390x/kvm: enable guarded storage
s390x/kvm: Enable KSS facility for nested virtualization
s390x/cpumodel: add esop/esop2 to z12 model
s390x/cpumodel: we are always in zarchitecture mode
s390x/cpumodel: wire up new hardware features
s390x/flic: migrate ais states
s390x/cpumodel: add zpci, aen and ais facilities
s390x: initialize cpu firstly
pc-bios/s390: rebuild s390-ccw.img
pc-bios/s390: add s390-netboot.img
pc-bios/s390-ccw: Link libnet into the netboot image and do the TFTP load
pc-bios/s390-ccw: Add virtio-net driver code
pc-bios/s390-ccw: Add core files for the network bootloading program
roms/SLOF: Update submodule to latest status
pc-bios/s390-ccw: Add code for virtio feature negotiation
pc-bios/s390-ccw: Remove unused structs from virtio.h
pc-bios/s390-ccw: Move byteswap functions to a separate header
pc-bios/s390-ccw: Add a write() function for stdio
...
* remotes/bonzini/tags/for-upstream: (55 commits)
spapr_rng: Convert to DEFINE_PROP_LINK
cpu: Convert to DEFINE_PROP_LINK
mips_cmgcr: Convert to DEFINE_PROP_LINK
ivshmem: Convert to DEFINE_PROP_LINK
dimm: Convert to DEFINE_PROP_LINK
virtio-crypto: Convert to DEFINE_PROP_LINK
virtio-rng: Convert to DEFINE_PROP_LINK
virtio-scsi: Convert to DEFINE_PROP_LINK
virtio-blk: Convert to DEFINE_PROP_LINK
qdev: Add const qualifier to PropertyInfo definitions
qmp: Use ObjectProperty.type if present
qdev: Introduce DEFINE_PROP_LINK
qdev: Introduce PropertyInfo.create
qom: enforce readonly nature of link's check callback
translate-all: remove redundant !tcg_enabled check in dump_exec_info
vl: fix breakage of -tb-size
nbd: Implement NBD_INFO_BLOCK_SIZE on client
nbd: Implement NBD_INFO_BLOCK_SIZE on server
nbd: Implement NBD_OPT_GO on client
nbd: Implement NBD_OPT_GO on server
...
Fan Zhang [Wed, 15 Feb 2017 03:47:49 +0000 (04:47 +0100)]
s390x/kvm: enable guarded storage
Introduce guarded storage support for KVM guests on s390.
We need to enable the capability, extend machine check validity,
sigp store-additional-status-at-address, and migration.
The feature is fenced for older machine type versions.
Jason J. Herne [Mon, 10 Apr 2017 13:39:00 +0000 (09:39 -0400)]
s390x/cpumodel: add esop/esop2 to z12 model
Add esop and esop2 features to z12 model where esop2 was originally
introduced. Disable esop and esop2 when using compatibility machine
v2.9 or earlier.
Jason J. Herne [Thu, 1 Sep 2016 08:56:49 +0000 (10:56 +0200)]
s390x/cpumodel: we are always in zarchitecture mode
In QEMU, a guest VCPU always started in and never was able to leave
z/Architecture mode. Now we have an architected way of showing this
condition.
The SIGP SET ARCHITECTURE instruction is simply rejected. Linux as guest
seems to not care about the return value, which is a good thing
The new handling is just like already being in z/Architecture mode.
We'll not try to fake absence of this facility, but still not indicate
the facility in case some strange CPU model turned z/Architecture off
completely (which doesn't work either way but let's us see how a
guest would react on a lack of this facility).
Yi Min Zhao [Tue, 16 May 2017 10:58:44 +0000 (18:58 +0800)]
s390x/flic: migrate ais states
During migration we should transfer ais states to the target guest.
This patch introduces a subsection to kvm_s390_flic_vmstate and new
vmsd for qemu_flic. The ais states need to be migrated only when
ais is supported.
Yi Min Zhao [Wed, 14 Jun 2017 05:25:58 +0000 (13:25 +0800)]
s390x/cpumodel: add zpci, aen and ais facilities
zPCI instructions and facilities are available since IBM zEnterprise
EC12. To support z/PCI in QEMU we enable zpci, aen and ais facilities
starting with zEC12 GA1. And we always set zpci and aen bits in max cpu
model. Later they might be switched off due to applied real cpu model.
For ais bit, we only provide it in the full cpu model beginning with
zEC12 and defer its enablement in the default cpu model to a later point
in time. At the same time, disable them for 2.9 and older machines.
Because of introducing AIS facility, we could check if it's enabled to
initialize flic->ais_supported with the real value.
4b996d0 pc-bios/s390-ccw: Link libnet into the netboot image and do the TFTP load e6879a6 pc-bios/s390-ccw: Add virtio-net driver code 766500f pc-bios/s390-ccw: Add core files for the network bootloading program f807e55 pc-bios/s390-ccw: Add code for virtio feature negotiation b4e3b4f pc-bios/s390-ccw: Remove unused structs from virtio.h dd3dc5e pc-bios/s390-ccw: Move byteswap functions to a separate header a20b4fe pc-bios/s390-ccw: Add a write() function for stdio 262e07c pc-bios/s390-ccw: Move virtio-block related functions into a separate file 7438d32 pc-bios/s390-ccw: Move ebc2asc to sclp.c 8760bad pc-bios/s390-ccw: Move libc functions to separate header c68f450 pc-bios/s390-ccw: use STRIP variable in Makefile
It's already possible to do a network boot of an s390x guest with an
external netboot image based on a Linux installation, but it would
be much more convenient if the s390-ccw firmware supported network
booting right out of the box, without the need to assemble such an
external image first.
This is an s390-netboot.img that can be used for network booting.
You can download a combined kernel + initrd image via TFTP
by starting QEMU for example with:
Thomas Huth [Wed, 12 Jul 2017 12:49:51 +0000 (14:49 +0200)]
pc-bios/s390-ccw: Add core files for the network bootloading program
This is just a preparation for the next steps: Add a makefile and a
stripped down copy of pc-bios/s390-ccw/main.c as a basis for the network
bootloader program, linked against the libc from SLOF already (which we
will need for SLOF's libnet). The networking code is not included yet.
Thomas Huth [Wed, 12 Jul 2017 12:49:46 +0000 (14:49 +0200)]
pc-bios/s390-ccw: Add a write() function for stdio
The stdio functions from the SLOF libc need a write() function for
printing text to stdout/stderr. Let's implement this function by
refactoring the code from sclp_print().
Thomas Huth [Wed, 12 Jul 2017 12:49:44 +0000 (14:49 +0200)]
pc-bios/s390-ccw: Move ebc2asc to sclp.c
We will later need this array in a file that we will link to the
netboot code, too. Since there is some ebcdic conversion done
in sclp_get_loadparm_ascii(), the sclp.c file seems to be a good
candidate.
projects / qemu.git / log