Kevin Wolf [Tue, 30 Jul 2019 14:49:26 +0000 (16:49 +0200)]
iotests/118: Add -blockdev based tests
The code path for -device drive=<node-name> or without a drive=...
option for empty drives, which is supposed to be used with -blockdev
differs enough from the -drive based path with a user-owned
BlockBackend, so we want to test both paths at least for the basic tests
implemented by TestInitiallyFilled and TestInitiallyEmpty.
This would have caught the bug recently fixed for inserting read-only
nodes into a scsi-cd created without a drive=... option.
Kevin Wolf [Tue, 30 Jul 2019 14:25:55 +0000 (16:25 +0200)]
iotests/118: Create test classes dynamically
We're getting a ridiculous number of child classes of
TestInitiallyFilled and TestInitiallyEmpty that differ only in a few
attributes that we want to test in all combinations.
Instead of explicitly writing down every combination, let's use a loop
and create those classes dynamically.
We'll need some connection parameters to be available all the time to
implement nbd reconnect. So, let's refactor them: define additional
parameters in BDRVNBDState, drop them from function parameters, drop
nbd_client_init and separate options parsing instead from nbd_open.
To implement reconnect we need several states for the client:
CONNECTED, QUIT and two different CONNECTING states. CONNECTING states
will be added in the following patches. This patch implements CONNECTED
and QUIT.
QUIT means, that we should close the connection and fail all current
and further requests (like old quit = true).
CONNECTED means that connection is ok, we can send requests (like old
quit = false).
For receiving loop we use a comparison of the current state with QUIT,
because reconnect will be in the same loop, so it should be looping
until the end.
Opposite, for requests we use a comparison of the current state with
CONNECTED, as we don't want to send requests in future CONNECTING
states.
block/nbd: use non-blocking io channel for nbd negotiation
No reason to use blocking channel for negotiation and we'll benefit in
further reconnect feature, as qio_channel reads and writes will do
qemu_coroutine_yield while waiting for io completion.
block/nbd: split connection_co start out of nbd_client_connect
nbd_client_connect is going to be used from connection_co, so, let's
refactor nbd_client_connect in advance, leaving io channel
configuration all in nbd_client_connect.
This helps to avoid extra io, allocations and memory copying.
We assume here that CMD_CACHE is always used with copy-on-read, as
otherwise it's a noop.
Thomas Huth [Tue, 23 Jul 2019 19:22:39 +0000 (21:22 +0200)]
tests/libqtest: Make qmp_assert_success() independent from global_qtest
The normal libqtest library functions should never depend on global_qtest.
Pass in the test state via parameter instead. And while we're at it,
also rename this function to qtest_qmp_assert_success() to make it clear
that it is part of libqtest.
Thomas Huth [Mon, 22 Jul 2019 15:10:55 +0000 (17:10 +0200)]
tests/libqtest: Make qtest_qmp_device_add/del independent from global_qtest
Generic library functions like qtest_qmp_device_add() and _del()
should not depend on the global_qtest variable. Pass the test
state via parameter instead.
Thomas Huth [Mon, 22 Jul 2019 14:17:38 +0000 (16:17 +0200)]
tests/libqtest: Remove unused function hmp()
No test is using hmp() anymore, and since this function uses the disliked
global_qtest variable, we should also make sure that nobody adds new code
with this function again. qtest_hmp() should be used instead.
Thomas Huth [Thu, 18 Jul 2019 15:08:51 +0000 (17:08 +0200)]
tests/libqos: Make virtio-pci code independent from global_qtest
The libqos library functions should never depend on global_qtest,
since these functions might be used in tests that track multiple
test states. So let's use the test state of the QPCIDevice instead.
Thomas Huth [Sat, 18 May 2019 08:23:24 +0000 (10:23 +0200)]
tests/libqos: Make generic virtio code independent from global_qtest
The libqos library functions should never depend on global_qtest,
since these functions might be used in tests that track multiple
test states. Pass around a pointer to the QTestState instead.
This patch is to reduce the number of Valgrind report messages about
using uninitialized memory with the null-co driver. It helps to filter
real memory issues and is the same work done for the iotests with the
commit ID a6862418fec4072.
Wei Yang [Mon, 5 Aug 2019 05:31:46 +0000 (13:31 +0800)]
migration/postcopy: use mis->bh instead of allocating a QEMUBH
For migration incoming side, it either quit in precopy or postcopy. It
is safe to use the mis->bh for both instead of allocating a dedicated
QEMUBH for postcopy.
Add qemu_file_update_transfer for just update bytes_xfer for speed
limitation. This will be used for further migration feature such as
multifd migration.
Ivan Ren [Fri, 2 Aug 2019 10:18:41 +0000 (18:18 +0800)]
migration: always initialise ram_counters for a new migration
This patch fix a multifd migration bug in migration speed calculation, this
problem can be reproduced as follows:
1. start a vm and give a heavy memory write stress to prevent the vm be
successfully migrated to destination
2. begin a migration with multifd
3. migrate for a long time [actually, this can be measured by transferred bytes]
4. migrate cancel
5. begin a new migration with multifd, the migration will directly run into
migration_completion phase
Reason as follows:
Migration update bandwidth and s->threshold_size in function
migration_update_counters after BUFFER_DELAY time:
In multifd migration, migration_total_bytes function return
qemu_ftell(s->to_dst_file) + ram_counters.multifd_bytes.
s->iteration_initial_bytes will be initialized to 0 at every new migration,
but ram_counters is a global variable, and history migration data will be
accumulated. So if the ram_counters.multifd_bytes is big enough, it may lead
pending_size >= s->threshold_size become false in migration_iteration_run
after the first migration_update_counters.
Wei Yang [Tue, 6 Aug 2019 00:36:45 +0000 (08:36 +0800)]
hmp: Remove migration capabilities from "info migrate"
With the growth of migration capabilities, it is not proper to display
them in "info migrate". Users are recommended to use "info
migrate_capabiltiies" to list them.
Currently, we allocate/deallocate PostcopyDiscardState for each RAMBlock
on sending discard information to destination. This is not necessary and
the same data area could be reused for each RAMBlock.
This patch defines PostcopyDiscardState a static variable. By doing so:
1) avoid memory allocation and deallocation to the system
2) avoid potential failure of memory allocation
3) hide some details for their users
Wei Yang [Thu, 18 Jul 2019 06:42:57 +0000 (14:42 +0800)]
migration: equation is more proper than and to check LOADVM_QUIT
LOADVM_QUIT allows a command to quit all layers of nested loadvm loops,
while current return value check is not that proper even it works now.
Current return value check "ret & LOADVM_QUIT" would return true if
bit[0] is 1. This would be true when ret is -1 which is used to indicate
an error of handling a command.
Since there is only one place return LOADVM_QUIT and no other
combination of return value, use "ret == LOADVM_QUIT" would be more
proper.
Wei Yang [Thu, 18 Jul 2019 08:37:47 +0000 (16:37 +0800)]
migration/postcopy: start_postcopy could be true only when migrate_postcopy() return true
There is only one place to set start_postcopy to true,
qmp_migrate_start_postcopy(), which make sure start_postcopy could be
set to true when migrate_postcopy() return true.
At some point vmxnet3 live migration stopped working and git-bisect
didn't help finding a working version.
The issue is the PCI configuration space is not being migrated
successfully and MSIX remains masked at destination.
Remove the migration differentiation between PCI and PCIe since
the logic resides now inside VMSTATE_PCI_DEVICE.
Remove also the VMXNET3_COMPAT_FLAG_DISABLE_PCIE based differentiation
since at 'realize' time is decided if the device is PCI or PCIe,
then the above macro is enough.
Use the opportunity to move to the standard VMSTATE_MSIX
instead of the deprecated SaveVMHandlers.
Currently, there is no information about error if outgoing migration was failed
because of file channel errors.
Example (QMP session):
-> { "execute": "migrate", "arguments": { "uri": "exec:head -c 1" }}
<- { "return": {} }
...
-> { "execute": "query-migrate" }
<- { "return": { "status": "failed" }} // There is not error's description
And even in the QEMU's output there is nothing.
This patch
1) Adds errp for the most of QEMUFileOps
2) Adds qemu_file_get_error_obj/qemu_file_set_error_obj
3) And finally using of qemu_file_get_error_obj in migration.c
And now, the status for the mentioned fail will be:
-> { "execute": "query-migrate" }
<- { "return": { "status": "failed",
"error-desc": "Unable to write to command: Broken pipe" }}
Here's a very, very last minute pull request for qemu-4.1. This fixes
two nasty bugs with the XIVE interrupt controller in "dual" mode
(where the guest decides which interrupt controller it wants to use).
One occurs when resetting the guest while I/O is active, and the other
with migration of hotplugged CPUs.
The timing here is very unfortunate. Alas, we only spotted these bugs
very late, and I was sick last week, delaying analysis and fix even
further.
This series hasn't had nearly as much testing as I'd really like, but
I'd still like to squeeze it into qemu-4.1 if possible, since
definitely fixing two bad bugs seems like an acceptable tradeoff for
the risk of introducing different bugs.
Cédric Le Goater [Tue, 13 Aug 2019 06:48:53 +0000 (08:48 +0200)]
spapr/xive: Fix migration of hot-plugged CPUs
The migration sequence of a guest using the XIVE exploitation mode
relies on the fact that the states of all devices are restored before
the machine is. This is not true for hot-plug devices such as CPUs
which state come after the machine. This breaks migration because the
thread interrupt context registers are not correctly set.
Fix migration of hotplugged CPUs by restoring their context in the
'post_load' handler of the XiveTCTX model.
David Gibson [Tue, 13 Aug 2019 05:59:18 +0000 (15:59 +1000)]
spapr: Reset CAS & IRQ subsystem after devices
This fixes a nasty regression in qemu-4.1 for the 'pseries' machine,
caused by the new "dual" interrupt controller model. Specifically,
qemu can crash when used with KVM if a 'system_reset' is requested
while there's active I/O in the guest.
The problem is that in spapr_machine_reset() we:
1. Reset the CAS vector state
spapr_ovec_cleanup(spapr->ov5_cas);
2. Reset all devices
qemu_devices_reset()
3. Reset the irq subsystem
spapr_irq_reset();
However (1) implicitly changes the interrupt delivery mode, because
whether we're using XICS or XIVE depends on the CAS state. We don't
properly initialize the new irq mode until (3) though - in particular
setting up the KVM devices.
During (2), we can temporarily drop the BQL allowing some irqs to be
delivered which will go to an irq system that's not properly set up.
Specifically, if the previous guest was in (KVM) XIVE mode, the CAS
reset will put us back in XICS mode. kvm_kernel_irqchip() still
returns true, because XIVE was using KVM, however XICs doesn't have
its KVM components intialized and kernel_xics_fd == -1. When the irq
is delivered it goes via ics_kvm_set_irq() which assert()s that
kernel_xics_fd != -1.
This change addresses the problem by delaying the CAS reset until
after the devices reset. The device reset should quiesce all the
devices so we won't get irqs delivered while we mess around with the
IRQ. The CAS reset and irq re-initialize should also now be under the
same BQL critical section so nothing else should be able to interrupt
it either.
We also move the spapr_irq_msi_reset() used in one of the legacy irq
modes, since it logically makes sense at the same point as the
spapr_irq_reset() (it's essentially an equivalent operation for older
machine types). Since we don't need to switch between different
interrupt controllers for those old machine types it shouldn't
actually be broken in those cases though.
Cc: Cédric Le Goater <[email protected]> Fixes: b2e22477 "spapr: add a 'reset' method to the sPAPR IRQ backend" Fixes: 13db0cd9 "spapr: introduce a new sPAPR IRQ backend supporting
XIVE and XICS" Signed-off-by: David Gibson <[email protected]>
Gerd Hoffmann [Mon, 12 Aug 2019 06:52:21 +0000 (08:52 +0200)]
display/bochs: fix pcie support
Set QEMU_PCI_CAP_EXPRESS unconditionally in init(), then clear it in
realize() in case the device is not connected to a PCIe bus.
This makes sure the pci config space allocation is big enough, so
accessing the PCIe extended config space doesn't overflow the pci
config space buffer.
PCI(e) config space is guest writable. Writes are limited by
write mask (which probably is also filled with random stuff),
so the guest can only flip enabled bits. But I suspect it
still might be exploitable, so rather serious because it might
be a host escape for the guest. On the other hand the device
is probably not yet in widespread use.
(For a QEMU version without this commit, a mitigation for the
bug is available: use "-device bochs-display" as a conventional pci
device only.)
Cornelia Huck [Tue, 6 Aug 2019 11:58:19 +0000 (13:58 +0200)]
compat: disable edid on virtio-gpu base device
'edid' is a property of the virtio-gpu base device, so turning
it off on virtio-gpu-pci is not enough (it misses -ccw). Turn
it off on the base device instead.
Peter Maydell [Tue, 6 Aug 2019 12:40:31 +0000 (13:40 +0100)]
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-08-06' into staging
Block patches for 4.1.0-rc4:
- Fix the backup block job when using copy offloading
- Fix the mirror block job when using the write-blocking copy mode
- Fix incremental backups after the image has been grown with the
respective bitmap attached to it
* remotes/maxreitz/tags/pull-block-2019-08-06:
block/backup: disable copy_range for compressed backup
iotests: Test unaligned blocking mirror write
mirror: Only mirror granularity-aligned chunks
iotests: Test incremental backup after truncation
util/hbitmap: update orig_size on truncate
iotests: Test backup job with two guest writes
backup: Copy only dirty areas
Max Reitz [Mon, 5 Aug 2019 15:33:08 +0000 (17:33 +0200)]
mirror: Only mirror granularity-aligned chunks
In write-blocking mode, all writes to the top node directly go to the
target. We must only mirror chunks of data that are aligned to the
job's granularity, because that is how the dirty bitmap works.
Therefore, the request alignment for writes must be the job's
granularity (in write-blocking mode).
Unfortunately, this forces all reads and writes to have the same
granularity (we only need this alignment for writes to the target, not
the source), but that is something to be fixed another time.
Without this, hbitmap_next_zero and hbitmap_next_dirty_area are broken
after truncate. So, orig_size is broken since it's introduction in 76d570dc495c56bb.
Max Reitz [Thu, 1 Aug 2019 17:39:00 +0000 (19:39 +0200)]
iotests: Test backup job with two guest writes
Perform two guest writes to not yet backed up areas of an image, where
the former touches an inner area of the latter.
Before HEAD^, copy offloading broke this in two ways:
(1) The target image differs from the reference image (what the source
was when the backup started).
(2) But you will not see that in the failing output, because the job
offset is reported as being greater than the job length. This is
because one cluster is copied twice, and thus accounted for twice,
but of course the job length does not increase.
Max Reitz [Thu, 1 Aug 2019 17:38:59 +0000 (19:38 +0200)]
backup: Copy only dirty areas
The backup job must only copy areas that the copy_bitmap reports as
dirty. This is always the case when using traditional non-offloading
backup, because it copies each cluster separately. When offloading the
copy operation, we sometimes copy more than one cluster at a time, but
we only check whether the first one is dirty.
Therefore, whenever copy offloading is possible, the backup job
currently produces wrong output when the guest writes to an area of
which an inner part has already been backed up, because that inner part
will be re-copied.
Olaf Hering [Thu, 30 May 2019 19:28:11 +0000 (21:28 +0200)]
Makefile: remove DESTDIR from firmware file content
The resulting firmware files should only contain the runtime path.
Fixes commit 26ce90fde5c ("Makefile: install the edk2 firmware images
and their descriptors")
Peter Maydell [Thu, 1 Aug 2019 10:57:42 +0000 (11:57 +0100)]
target/arm: Avoid bogus NSACR traps on M-profile without Security Extension
In Arm v8.0 M-profile CPUs without the Security Extension and also in
v7M CPUs, there is no NSACR register. However, the code we have to handle
the FPU does not always check whether the ARM_FEATURE_M_SECURITY bit
is set before testing whether env->v7m.nsacr permits access to the
FPU. This means that for a CPU with an FPU but without the Security
Extension we would always take a bogus fault when trying to stack
the FPU registers on an exception entry.
We could fix this by adding extra feature bit checks for all uses,
but it is simpler to just make the internal value of nsacr 0xcff
("all non-secure accesses allowed"), since this is not guest
visible when the Security Extension is not present. This allows
us to continue to follow the Arm ARM pseudocode which takes a
similar approach. (In particular, in the v8.1 Arm ARM the register
is documented as reading as 0xcff in this configuration.)
ACS got added in 4.0 unconditionally, that broke older<->4.0 migration
where there was a PCIe root port.
Fix this by turning it off for 3.1 and older machines; note this
fixes compatibility for older QEMUs but breaks compatibility with 4.0
for older machine types.
machine type source qemu dest qemu
3.1 3.1 4.0 broken
3.1 3.1 4.1rc2 broken
3.1 3.1 4.1+this OK ++
3.1 4.0 4.1rc2 OK
3.1 4.0 4.1+this broken --
4.0 4.0 4.1rc2 OK
4.0 4.0 4.1+this OK
So we gain and lose; the consensus seems to be treat this as a
fix for older machine types.
ACS was added in 4.0 unconditionally, this breaks migration
compatibility.
Allow ACS to be disabled by adding a property that's
checked by pcie_root_port.
Unfortunately pcie-root-port doesn't have any instance data,
so there's no where for that flag to live, so stuff it into
PCIESlot.
Peter Maydell [Tue, 30 Jul 2019 13:25:22 +0000 (14:25 +0100)]
target/arm: Deliver BKPT/BRK exceptions to correct exception level
Most Arm architectural debug exceptions (eg watchpoints) are ignored
if the configured "debug exception level" is below the current
exception level (so for example EL1 can't arrange to get debug exceptions
for EL2 execution). Exceptions generated by the BRK or BPKT instructions
are a special case -- they must always cause an exception, so if
we're executing above the debug exception level then we
must take them to the current exception level.
This fixes a bug where executing BRK at EL2 could result in an
exception being taken at EL1 (which is strictly forbidden by the
architecture).
Kevin Wolf [Tue, 30 Jul 2019 13:37:08 +0000 (15:37 +0200)]
fdc: Fix inserting read-only media in empty drive
In order to insert a read-only medium (i.e. a read-only block node) to
the BlockBackend of a floppy drive, we must not have taken write
permissions on that BlockBackend, or the operation will fail with the
error message "Block node is read-only".
The device already takes care to remove all permissions when the medium
is ejected, but the state isn't correct if the drive is initially empty:
It uses blk_is_read_only() to check whether write permissions should be
taken, but this function returns false for empty BlockBackends in the
common case.
Fix floppy_drive_realize() to avoid taking write permissions if the
drive is empty.
Peter Maydell [Tue, 30 Jul 2019 11:25:34 +0000 (12:25 +0100)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches:
- scsi-cd: Fix inserting read-only media in empty drive
- block/copy-on-read: Fix permissions for inactive node
- Test case fixes
# gpg: Signature made Tue 30 Jul 2019 12:21:48 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <[email protected]>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
scsi-cd: Fix inserting read-only media in empty drive
block/copy-on-read: Fix permissions for inactive node Fixes: add read-zeroes to 051.out
tests/multiboot: Fix load address of test kernels
Kevin Wolf [Mon, 29 Jul 2019 16:33:33 +0000 (18:33 +0200)]
scsi-cd: Fix inserting read-only media in empty drive
scsi-disks decides whether it has a read-only device by looking at
whether the BlockBackend specified as drive=... is read-only. In the
case of an anonymous BlockBackend (with a node name specified in
drive=...), this is the read-only flag of the attached node. In the case
of an empty anonymous BlockBackend, it's always read-write because
nothing prevented it from being read-write.
This is a problem because scsi-cd would take write permissions on the
anonymous BlockBackend of an empty drive created without a drive=...
option. Using blockdev-insert-medium with a read-only node fails then
with the error message "Block node is read-only".
Fix scsi_realize() so that scsi-cd devices always take read-only
permissions on their BlockBackend instead.
Kevin Wolf [Mon, 29 Jul 2019 10:45:14 +0000 (12:45 +0200)]
block/copy-on-read: Fix permissions for inactive node
The copy-on-read drive must not request the WRITE_UNCHANGED permission
for its child if the node is inactive, otherwise starting a migration
destination with -incoming will fail because the child cannot provide
write access yet:
qemu-system-x86_64: -blockdev copy-on-read,file=img,node-name=cor: Block node is read-only
Earlier QEMU versions additionally ran into an abort() on the migration
source side: bdrv_inactivate_recurse() failed to update permissions.
This is silently ignored today because it was only supposed to loosen
restrictions. This is the symptom that was originally reported here:
Fixes: add read-zeroes to 051.out
The patch "iotests: Set read-zeroes on in null block driver for Valgrind"
with the commit ID a6862418fec4072 needs the change in 051.out when
compared against on the s390 system.
Kevin Wolf [Mon, 22 Jul 2019 09:26:15 +0000 (11:26 +0200)]
tests/multiboot: Fix load address of test kernels
While older toolchains produced binaries where the physical load address
of ELF segments was the same as the virtual address, newer versions seem
to choose a different physical address if it isn't specified explicitly.
The means that the test kernel doesn't use the right addresses to access
e.g. format strings any more and the whole output disappears, causing
all test cases to fail.
Fix this by specifying the physical load address of sections explicitly.
Peter Maydell [Tue, 30 Jul 2019 08:43:32 +0000 (09:43 +0100)]
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging
virtio, pc: fixes
A couple of last minute bugfixes.
Signed-off-by: Michael S. Tsirkin <[email protected]>
# gpg: Signature made Mon 29 Jul 2019 22:13:22 BST
# gpg: using RSA key 281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <[email protected]>" [full]
# gpg: aka "Michael S. Tsirkin <[email protected]>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469
* remotes/mst/tags/for_upstream:
pc-dimm: fix crash when invalid slot number is used
Revert "hw: report invalid disable-legacy|modern usage for virtio-1-only devs"
Revert "Revert "globals: Allow global properties to be optional""
Revert "hw: report invalid disable-legacy|modern usage for virtio-1-only devs"
This reverts commit f2784eed306449c3d04a71a05ed6463b8289aedf
since that accidentally removes the PCIe capabilities from virtio
devices because virtio_pci_dc_realize is called before the new 'mode'
flag is set.
Peter Maydell [Mon, 29 Jul 2019 11:04:53 +0000 (12:04 +0100)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
# gpg: Signature made Mon 29 Jul 2019 09:30:48 BST
# gpg: using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <[email protected]>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* remotes/jasowang/tags/net-pull-request:
net/colo-compare.c: Fix memory leak and code style issue.
net: tap: replace snprintf with g_strdup_printf calls
qemu-bridge-helper: move repeating code in parse_acl_file
qemu-bridge-helper: restrict interface name to IFNAMSIZ
e1000: don't raise interrupt in pre_save()
projects / qemu.git / log