Vadim Galitsyn [Mon, 23 Oct 2017 15:13:10 +0000 (17:13 +0200)]
tests: test-hmp: print command execution result
Provide HMP monitor command execution result as it would be seen
by user who established an HMP monitor session.
Currently many commands may silently fail without any sign of that.
This patch let this info to be printed once test is running in
verbose mode.
For the future it might be useful to fail the test if command has
failed, however it would require a bit of rework inside test
engine itself.
A simple example of silent failure without reporting it would to
add some non-existent HMP command into 'hmp_cmds' list. In this case
test will report it successfully passed without error.
Thomas Huth [Thu, 30 Nov 2017 20:19:00 +0000 (21:19 +0100)]
hmp-commands: Remove the deprecated usb_add and usb_del
It's easy to use device_add and device_del as replacement instead.
The usb_add and usb_del commands are deprecated since QEMU 2.10,
and nobody complained that they are still needed, so let's get rid
of them now to make the HMP interface a little bit less overloaded.
Peter Maydell [Mon, 11 Dec 2017 15:42:44 +0000 (15:42 +0000)]
target/arm: Generate UNDEF for 32-bit Thumb2 insns
The refactoring of commit 296e5a0a6c3935 has a nasty bug:
it accidentally dropped the generation of code to raise
the UNDEF exception when disas_thumb2_insn() returns nonzero.
This means that 32-bit Thumb2 instruction patterns that
ought to UNDEF just act like nops instead. This is likely
to break any number of things, including the kernel's "disable
the FPU and use the UNDEF exception to identify when to turn
it back on again" trick.
Eric Farman [Fri, 1 Dec 2017 15:15:38 +0000 (16:15 +0100)]
vhost-scsi: add missing virtqueue_size parameter
Commit 5c0919d02066 ("virtio-scsi: Add virtqueue_size parameter allowing
virtqueue size to be set.") introduced a new parameter to virtio-scsi.
Later, commit 920036106044 ("vhost-user-scsi: add missing virtqueue_size
param") added that parameter to the new vhost-user-scsi interface but
neglected the existing vhost-scsi interface it was built on.
Apply the same change to vhost-scsi, so that we can boot a guest with
a device defined. This also avoids crashing a guest when hotplugging
a vhost-scsi device.
Peter Maydell [Tue, 5 Dec 2017 10:00:48 +0000 (10:00 +0000)]
Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-2.11-20171205' into staging
ppc patch queue 2017-12-05
Alas, this is yet another fix for ppc that I think it's worth
squeezing into 2.11. It's a really ugly fix for some pretty ugly
code, but it does seem to address a real problem. It's also a problem
that's appeared relatively recently, since it was either created by,
or made much easier to trigger by, by the merge of MTTCG.
The "00000004" is CPU_INTERRUPT_EXITTB yet the code calls
cpu_interrupt(cs, CPU_INTERRUPT_HARD) ("00000002") in this function
just before the log message. Something is causing the HARD bit setting
to get lost.
The knock on effect of losing that bit is the decrementer timer interrupts
don't get delivered which causes the guest to sit idle in its idle handler
and 'hang'.
The issue occurs due to races from code which sets CPU_INTERRUPT_EXITTB.
Rather than poking directly into cs->interrupt_request, that code needs to:
a) hold BQL
b) use the cpu_interrupt() helper
This patch fixes the call sites to do this, fixing the hang. The calls
are made from a variety of contexts so a helper function is added to handle
the necessary locking. This can likely be improved and optimised in the future
but it ensures the code is correct and doesn't lockup as it stands today.
Alberto Garcia [Wed, 29 Nov 2017 17:56:34 +0000 (19:56 +0200)]
blockjob: Make block_job_pause_all() keep a reference to the jobs
Starting from commit 40840e419be31e6a32e6ea24511c74b389d5e0e4 we are
pausing all block jobs during bdrv_reopen_multiple() to prevent any of
them from finishing and removing nodes from the graph while they are
being reopened.
It turns out that pausing a block job doesn't necessarily prevent it
from finishing: a paused block job can still run its exit function
from the main loop and call block_job_completed(). The mirror block
job in particular always goes to the main loop while it is paused (by
virtue of the bdrv_drained_begin() call in mirror_run()).
Destroying a paused block job during bdrv_reopen_multiple() has two
consequences:
1) The references to the nodes involved in the job are released,
possibly destroying some of them. If those nodes were in the
reopen queue this would trigger the problem originally described
in commit 40840e419be, crashing QEMU.
2) At the end of bdrv_reopen_multiple(), bdrv_drain_all_end() would
not be doing all necessary bdrv_parent_drained_end() calls.
I can reproduce problem 1) easily with iotest 030 by increasing
STREAM_BUFFER_SIZE from 512KB to 8MB in block/stream.c, or by tweaking
the iotest like in this example:
This patch keeps an additional reference to all block jobs between
block_job_pause_all() and block_job_resume_all(), guaranteeing that
they are kept alive.
Peter Maydell [Mon, 4 Dec 2017 13:08:13 +0000 (13:08 +0000)]
Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging
pc, pci, virtio: fixes for rc3
A bunch of fixes all over the place.
Signed-off-by: Michael S. Tsirkin <[email protected]>
# gpg: Signature made Fri 01 Dec 2017 17:06:33 GMT
# gpg: using RSA key 0x281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <[email protected]>"
# gpg: aka "Michael S. Tsirkin <[email protected]>"
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17 0970 C350 3912 AFBE 8E67
# Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA 8A0D 281F 0DB8 D28D 5469
* remotes/mst/tags/for_upstream:
pc: fix crash on attempted cpu unplug
virtio: check VirtQueue Vring object is set
vhost: fix error check in vhost_verify_ring_mappings()
dump-guest-memory.py: fix No symbol "vmcoreinfo_find"
vhost: restore avail index from vring used index on disconnection
virtio: Add queue interface to restore avail index from vring used index
i386/msi: Correct mask of destination ID in MSI address
Peter Maydell [Mon, 4 Dec 2017 11:27:53 +0000 (11:27 +0000)]
Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-2.11-20171204' into staging
ppc patch queue 2017-12-04
We are, alas, not yet to the bottom of ppc bugs. This pull request
fixes several more. I believe they're important enough to include in
2.11. despite the late date.
David Gibson [Fri, 1 Dec 2017 05:05:33 +0000 (16:05 +1100)]
spapr: Include "pre-plugged" DIMMS in ram size calculation at reset
At guest reset time, we allocate a hash page table (HPT) for the guest
based on the guest's RAM size. If dynamic HPT resizing is not available we
use the maximum RAM size, if it is we use the current RAM size.
But the "current RAM size" calculation is incorrect - we just use the
"base" ram_size from the machine structure. This doesn't include any
pluggable DIMMs that are already plugged at reset time.
This means that if you try to start a 'pseries' machine with a DIMM
specified on the command line that's much larger than the "base" RAM size,
then the guest will get a woefully inadequate HPT. This can lead to a
guest freeze during boot as it runs out of HPT space during initial MMU
setup.
which is caused by pcms->acpi_dev == NULL due to ACPI support
being disabled.
Considering that ACPI support is necessary for unplug to work,
check that it's enabled and fail unplug request gracefully
if no acpi device were found.
Greg Kurz [Thu, 30 Nov 2017 21:39:59 +0000 (22:39 +0100)]
vhost: fix error check in vhost_verify_ring_mappings()
Since commit f1f9e6c5 "vhost: adapt vhost_verify_ring_mappings() to
virtio 1 ring layout", we check the mapping of each part (descriptor
table, available ring and used ring) of each virtqueue separately.
The checking of a part is done by the vhost_verify_ring_part_mapping()
function: it returns either 0 on success or a negative errno if the
part cannot be mapped at the same place.
Unfortunately, the vhost_verify_ring_mappings() function checks its
return value the other way round. It means that we either:
- only verify the descriptor table of the first virtqueue, and if it
is valid we ignore all the other mappings
- or ignore all broken mappings until we reach a valid one
ie, we only raise an error if all mappings are broken, and we consider
all mappings are valid otherwise (false success), which is obviously
wrong.
This patch ensures that vhost_verify_ring_mappings() only returns
success if ALL mappings are okay.
Chao Gao [Fri, 17 Nov 2017 06:24:23 +0000 (14:24 +0800)]
i386/msi: Correct mask of destination ID in MSI address
According to SDM 10.11.1, only [19:12] bits of MSI address are
Destination ID, change the mask to avoid ambiguity for VT-d spec
has used the bit 4 to indicate a remappable interrupt request.
The msr invalidation code (commits 993eb and 2360b) inverts all
bits except MSR_TGPR and MSR_HVB. On non PowerPC 601 processors
this leads to incorrect change of excp_prefix in hreg_store_msr()
function. The problem is that new msr value get multiplied by msr_mask
and inverted msr does not, thus values of MSR_EP bit in new msr value
and inverted msr are distinct, so that excp_prefix changes but should
not.
Paolo Bonzini [Wed, 29 Nov 2017 10:25:13 +0000 (11:25 +0100)]
blockjob: reimplement block_job_sleep_ns to allow cancellation
This reverts the effects of commit 4afeffc857 ("blockjob: do not allow
coroutine double entry or entry-after-completion", 2017-11-21)
This fixed the symptom of a bug rather than the root cause. Canceling the
wait on a sleeping blockjob coroutine is generally fine, we just need to
make it work correctly across AioContexts. To do so, use a QEMUTimer
that calls block_job_enter. Use a mutex to ensure that block_job_enter
synchronizes correctly with block_job_sleep_ns.
Paolo Bonzini [Wed, 29 Nov 2017 10:25:12 +0000 (11:25 +0100)]
blockjob: introduce block_job_do_yield
Hide the clearing of job->busy in a single function, and set it
in block_job_enter. This lets block_job_do_yield verify that
qemu_coroutine_enter is not used while job->busy = false.
Paolo Bonzini [Wed, 29 Nov 2017 10:25:11 +0000 (11:25 +0100)]
blockjob: remove clock argument from block_job_sleep_ns
All callers are using QEMU_CLOCK_REALTIME, and it will not be possible to
support more than one clock when block_job_sleep_ns switches to a single
timer stored in the BlockJob struct.
Kevin Wolf [Wed, 29 Nov 2017 10:25:10 +0000 (11:25 +0100)]
block: Expect graph changes in bdrv_parent_drained_begin/end
The .drained_begin/end callbacks can (directly or indirectly via
aio_poll()) cause block nodes to be removed or the current BdrvChild to
point to a different child node.
Use QLIST_FOREACH_SAFE() to make sure we don't access invalid
BlockDriverStates or accidentally continue iterating the parents of the
new child node instead of the node we actually came from.
Alberto Garcia [Tue, 28 Nov 2017 14:53:27 +0000 (16:53 +0200)]
blockjob: Remove the job from the list earlier in block_job_unref()
When destroying a block job in block_job_unref() we should remove it
from the job list before calling block_job_remove_all_bdrv().
This is because removing the BDSs can trigger an aio_poll() and wake
up other jobs that might attempt to use the block job list. If that
happens the job we're currently destroying should not be in that list
anymore.
Peter Maydell [Tue, 28 Nov 2017 13:12:48 +0000 (13:12 +0000)]
Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2017-11-28' into staging
nbd patches for 2017-11-28
Eric Blake - 0/2 fix two NBD server CVEs
# gpg: Signature made Tue 28 Nov 2017 12:58:29 GMT
# gpg: using RSA key 0xA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <[email protected]>"
# gpg: aka "Eric Blake (Free Software Programmer) <[email protected]>"
# gpg: aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A
* remotes/ericb/tags/pull-nbd-2017-11-28:
nbd/server: CVE-2017-15118 Stack smash on large export name
nbd/server: CVE-2017-15119 Reject options larger than 32M
Eric Blake [Wed, 22 Nov 2017 21:07:22 +0000 (15:07 -0600)]
nbd/server: CVE-2017-15118 Stack smash on large export name
Introduced in commit f37708f6b8 (2.10). The NBD spec says a client
can request export names up to 4096 bytes in length, even though
they should not expect success on names longer than 256. However,
qemu hard-codes the limit of 256, and fails to filter out a client
that probes for a longer name; the result is a stack smash that can
potentially give an attacker arbitrary control over the qemu
process.
The smash can be easily demonstrated with this client:
$ qemu-io f raw nbd://localhost:10809/$(printf %3000d 1 | tr ' ' a)
If the qemu NBD server binary (whether the standalone qemu-nbd, or
the builtin server of QMP nbd-server-start) was compiled with
-fstack-protector-strong, the ability to exploit the stack smash
into arbitrary execution is a lot more difficult (but still
theoretically possible to a determined attacker, perhaps in
combination with other CVEs). Still, crashing a running qemu (and
losing the VM) is bad enough, even if the attacker did not obtain
full execution control.
Eric Blake [Wed, 22 Nov 2017 22:25:16 +0000 (16:25 -0600)]
nbd/server: CVE-2017-15119 Reject options larger than 32M
The NBD spec gives us permission to abruptly disconnect on clients
that send outrageously large option requests, rather than having
to spend the time reading to the end of the option. No real
option request requires that much data anyways; and meanwhile, we
already have the practice of abruptly dropping the connection on
any client that sends NBD_CMD_WRITE with a payload larger than 32M.
For comparison, nbdkit drops the connection on any request with
more than 4096 bytes; however, that limit is probably too low
(as the NBD spec states an export name can theoretically be up
to 4096 bytes, which means a valid NBD_OPT_INFO could be even
longer) - even if qemu doesn't permit exports longer than 256
bytes.
It could be argued that a malicious client trying to get us to
read nearly 4G of data on a bad request is a form of denial of
service. In particular, if the server requires TLS, but a client
that does not know the TLS credentials sends any option (other
than NBD_OPT_STARTTLS or NBD_OPT_EXPORT_NAME) with a stated
payload of nearly 4G, then the server was keeping the connection
alive trying to read all the payload, tying up resources that it
would rather be spending on a client that can get past the TLS
handshake. Hence, this warranted a CVE.
Present since at least 2.5 when handling known options, and made
worse in 2.6 when fixing support for NBD_FLAG_C_FIXED_NEWSTYLE
to handle unknown options.
sockets: avoid crash when cleaning up sockets for an invalid FD
If socket_listen_cleanup is passed an invalid FD, then querying the socket
local address will fail. We must thus be prepared for the returned addr to
be NULL
Peter Maydell [Tue, 28 Nov 2017 10:03:26 +0000 (10:03 +0000)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
# gpg: Signature made Tue 28 Nov 2017 03:58:11 GMT
# gpg: using RSA key 0xEF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <[email protected]>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* remotes/jasowang/tags/net-pull-request:
virtio-net: don't touch virtqueue if vm is stopped
Jason Wang [Wed, 22 Nov 2017 09:57:19 +0000 (17:57 +0800)]
virtio-net: don't touch virtqueue if vm is stopped
Guest state should not be touched if VM is stopped, unfortunately we
didn't check running state and tried to drain tx queue unconditionally
in virtio_net_set_status(). A crash was then noticed as a migration
destination when user type quit after virtqueue state is loaded but
before region cache is initialized. In this case,
virtio_net_drop_tx_queue_data() tries to access the uninitialized
region cache.
Fix this by only dropping tx queue data when vm is running.
QAPI & interop: Clarify events emitted by 'block-job-cancel'
When you cancel an in-progress 'mirror' job (or "active `block-commit`")
with QMP `block-job-cancel`, it emits the event: BLOCK_JOB_CANCELLED.
However, when `block-job-cancel` is issued *after* `drive-mirror` has
indicated (via the event BLOCK_JOB_READY) that the source and
destination have reached synchronization:
But this is expected behaviour, where the _COMPLETED event indicates
that synchronization has successfully ended (and the destination now has
a point-in-time copy, which is at the time of cancel).
So add a small note to this effect in 'block-core.json'. While at it,
also update the "Live disk synchronization -- drive-mirror and
blockdev-mirror" section in 'live-block-operations.rst'.
(Thanks: Max Reitz for reminding me of this caveat on IRC.)
* remotes/dgibson/tags/ppc-for-2.11-20171127:
target/ppc: Fix setting of cpu->compat_pvr on incoming migration
target/ppc: Move setting of patb_entry on hash table init
target/ppc: Fix setting of cpu->compat_pvr on incoming migration
cpu->compat_pvr is used to store the current compat mode of the cpu.
On the receiving side during incoming migration we check compatibility
with the compat mode by calling ppc_set_compat(). However we fail to set
the compat mode with the hypervisor since the "new" compat mode doesn't
differ from the current (due to a "cpu->compat_pvr != compat_pvr" check).
This means that kvm runs the vcpus without a compat mode, which is the
incorrect behaviour. The implication being that a compatibility mode
will never be in effect after migration.
To fix this so that the compat mode is correctly set with the
hypervisor, store the desired compat mode and reset cpu->compat_pvr to
zero before calling ppc_set_compat().
Fixes: 5dfaa532 ("ppc: fix ppc_set_compat() with KVM PR") Signed-off-by: Suraj Jitindar Singh <[email protected]> Signed-off-by: David Gibson <[email protected]>
target/ppc: Move setting of patb_entry on hash table init
The patb_entry is used to store the location of the process table in
guest memory. The msb is also used to indicate the mmu mode of the
guest, that is patb_entry & 1 << 63 ? radix_mode : hash_mode.
Currently we set this to zero in spapr_setup_hpt_and_vrma() since if
this function gets called then we know we're hash. However some code
paths, such as setting up the hpt on incoming migration of a hash guest,
call spapr_reallocate_hpt() directly bypassing this higher level
function. Since we assume radix if the host is capable this results in
the msb in patb_entry being left set so in spapr_post_load() we call
kvmppc_configure_v3_mmu() and tell the host we're radix which as
expected means addresses cannot be translated once we actually run the cpu.
To fix this move the zeroing of patb_entry into spapr_reallocate_hpt().
Peter Maydell [Thu, 23 Nov 2017 15:56:38 +0000 (15:56 +0000)]
osdep.h: Make TIME_MAX handle different time_t types
In our various supported host OSes, the time_t type may be either 32
or 64 bit, and could in theory also be either signed or unsigned.
Notably, in OpenBSD time_t is a 64 bit type even if 'long' is 32
bits, so using LONG_MAX for TIME_MAX is incorrect.
Use an approach suggested by Paolo Bonzini which calculates
the maximum value of the type rather than hardcoding it;
to do this we use the TYPE_MAXIMUM macro from Gnulib.
s390/kvm_virtio/linux-headers: remove traces of old virtio transport
We no longer support the old s390 transport, neither does the newest
Linux kernel. Remove it from the linux header script as well as the
s390x virtio code. We still should handle the VIRTIO_NOTIFY hypercall,
to tolerate early printk on older guest kernels without an sclp console.
We continue to ignore these events.
# gpg: Signature made Wed 22 Nov 2017 08:43:13 GMT
# gpg: using RSA key 0xF487EF185872D723
# gpg: Good signature from "Juan Quintela <[email protected]>"
# gpg: aka "Juan Quintela <[email protected]>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723
* remotes/juanquintela/tags/migration/20171122:
migration/ram.c: do not set 'postcopy_running' in POSTCOPY_INCOMING_END
migration, xen: Fix block image lock issue on live migration
Juan Quintela [Wed, 22 Nov 2017 08:41:57 +0000 (09:41 +0100)]
tcg: Fix compilation without TCG
Commit 27266271977c started to use tb_unlock() and tlb_set_dirty() on
non TCG code. Add the functions as stubs, so that builds with TCG
disabled continue to compile.
migration/ram.c: do not set 'postcopy_running' in POSTCOPY_INCOMING_END
When migrating a VM with 'migrate_set_capability postcopy-ram on'
a postcopy_state is set during the process, ending up with the
state POSTCOPY_INCOMING_END when the migration is over. This
postcopy_state is taken into account inside ram_load to check
how it will load the memory pages. This same ram_load is called when
in a loadvm command.
Inside ram_load, the logic to see if we're at postcopy_running state
is:
In the case where ram_load is executed and postcopy_state is
POSTCOPY_INCOMING_END, postcopy_running will be set to 'true' and
ram_load will behave like a postcopy is in progress. This scenario isn't
achievable in a migration but it is reproducible when executing
savevm/loadvm after migrating with 'postcopy-ram on', causing loadvm
to fail with Error -22:
Source:
(qemu) migrate_set_capability postcopy-ram on
(qemu) migrate tcp:127.0.0.1:4444
ubuntu1704-intel login: (qemu)
(qemu) savevm test1
(qemu) loadvm test1
Unknown combination of migration flags: 0x4 (postcopy mode)
error while loading state for instance 0x0 of device 'ram'
Error -22 while loading VM state
(qemu)
This patch fixes this problem by changing the existing logic for
postcopy_advised and postcopy_running in ram_load, making them
'false' if we're at POSTCOPY_INCOMING_END state.
Laurent Vivier [Tue, 21 Nov 2017 20:16:43 +0000 (21:16 +0100)]
ppc: fix VTB migration
Migration of a system under stress (for example, with
"stress-ng --numa 2") triggers on the destination
some kernel watchdog messages like:
NMI watchdog: BUG: soft lockup - CPU#0 stuck for 3489660870s!
NMI watchdog: BUG: soft lockup - CPU#1 stuck for 3489660884s!
This problem appears with the changes introduced by 42043e4 spapr: clock should count only if vm is running
I think this commit only triggers the problem.
Kernel computes the soft lockup duration using the
Virtual Timebase register (VTB), not using the Timebase
Register (TBR, the one 42043e4 stops).
It appears VTB is not migrated, so this patch adds it in
the list of the SPRs to migrate, and fixes the problem.
For the migration, I've tested a migration from qemu-2.8.0 and
pseries-2.8.0 to a patched master (qemu-2.11.0-rc1). The received
VTB is 0 (as is it not initialized by qemu-2.8.0), but the value
seems to be ignored by KVM and a non zero VTB is used by the kernel.
I have no explanation for that, but as the original problem appears
only with SMP system under stress I suspect some problems in KVM
(I think because VTB is shared by all threads of a core).
David Gibson [Mon, 20 Nov 2017 06:49:25 +0000 (17:49 +1100)]
spapr: Implement bug in spapr-vty device to be compatible with PowerVM
The spapr-vty device implements the PAPR defined virtual console,
which is also implemented by IBM's proprietary PowerVM hypervisor.
PowerVM's implementation has a bug where it inserts an extra \0 after
every \r going to the guest. Because of that Linux's guest side
driver has a workaround which strips \0 characters that appear
immediately after a \r.
That means that when running under qemu, sending a binary stream from
host to guest via spapr-vty which happens to include a \r\0 sequence
will get corrupted by that workaround.
To deal with that, this patch duplicates PowerVM's bug, inserting an
extra \0 after each \r. Ugly, but the best option available.
Thomas Huth [Mon, 20 Nov 2017 07:44:38 +0000 (08:44 +0100)]
hw/ppc/spapr: Fix virtio-scsi bootindex handling for LUNs >= 256
LUNs >= 256 have to be encoded with the so-called "flat space
addressing method" for virtio-scsi, where an additional bit has to
be set. SLOF already took care of this with the following commit:
https://git.qemu.org/?p=SLOF.git;a=commitdiff;h=f72a37713fea47da
(see https://bugzilla.redhat.com/show_bug.cgi?id=1431584 for details)
But QEMU does not use this encoding yet for device tree paths
that have to be handed over to SLOF to deal with the "bootindex"
property, so SLOF currently fails to boot from virtio-scsi devices
with LUNs >= 256 in the right boot order. Fix it by using the bit
to indicate the "flat space addressing method" for LUNs >= 256.
Anthony PERARD [Thu, 16 Nov 2017 15:14:19 +0000 (15:14 +0000)]
migration, xen: Fix block image lock issue on live migration
When doing a live migration of a Xen guest with libxl, the images for
block devices are locked by the original QEMU process, and this prevent
the QEMU at the destination to take the lock and the migration fail.
>From QEMU point of view, once the RAM of a domain is migrated, there is
two QMP commands, "stop" then "xen-save-devices-state", at which point a
new QEMU is spawned at the destination.
Release locks in "xen-save-devices-state" so the destination can takes
them, if it's a live migration.
This patch add the "live" parameter to "xen-save-devices-state" which
default to true so older version of libxenlight can work with newer
version of QEMU.
* remotes/cody/tags/block-pull-request:
qemu-iotest: add test for blockjob coroutine race condition
qemu-iotests: add option in common.qemu for mismatch only
coroutine: abort if we try to schedule or enter a pending coroutine
blockjob: do not allow coroutine double entry or entry-after-completion
Jeff Cody [Sat, 18 Nov 2017 03:27:09 +0000 (22:27 -0500)]
coroutine: abort if we try to schedule or enter a pending coroutine
The previous patch fixed a race condition, in which there were
coroutines being executing doubly, or after coroutine deletion.
We can detect common scenarios when this happens, and print an error
message and abort before we corrupt memory / data, or segfault.
This patch will abort if an attempt to enter a coroutine is made while
it is currently pending execution, either in a specific AioContext bh,
or pending execution via a timer. It will also abort if a coroutine
is scheduled, before a prior scheduled run has occurred.
We cannot rely on the existing co->caller check for recursive re-entry
to catch this, as the coroutine may run and exit with
COROUTINE_TERMINATE before the scheduled coroutine executes.
(This is the scenario that was occurring and fixed in the previous
patch).
This patch also re-orders the Coroutine struct elements in an attempt to
optimize caching.
Jeff Cody [Sat, 18 Nov 2017 03:26:16 +0000 (22:26 -0500)]
blockjob: do not allow coroutine double entry or entry-after-completion
When block_job_sleep_ns() is called, the co-routine is scheduled for
future execution. If we allow the job to be re-entered prior to the
scheduled time, we present a race condition in which a coroutine can be
entered recursively, or even entered after the coroutine is deleted.
The job->busy flag is used by blockjobs when a coroutine is busy
executing. The function 'block_job_enter()' obeys the busy flag,
and will not enter a coroutine if set. If we sleep a job, we need to
leave the busy flag set, so that subsequent calls to block_job_enter()
are prevented.
This changes the prior behavior of block_job_cancel() being able to
immediately wake up and cancel a job; in practice, this should not be an
issue, as the coroutine sleep times are generally very small, and the
cancel will occur the next time the coroutine wakes up.
This fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1508708
Peter Maydell [Tue, 21 Nov 2017 15:50:13 +0000 (15:50 +0000)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches for 2.11.0-rc2
# gpg: Signature made Tue 21 Nov 2017 15:09:12 GMT
# gpg: using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <[email protected]>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
iotests: Fix 176 on 32-bit host
block: Close a BlockDriverState completely even when bs->drv is NULL
block: Error out on load_vm with active dirty bitmaps
block: Add errp to bdrv_all_goto_snapshot()
block: Add errp to bdrv_snapshot_goto()
block: Don't request I/O permission with BDRV_O_NO_IO
block: Don't use BLK_PERM_CONSISTENT_READ for format probing
Developers sometimes mistakenly run 'make test' instead of 'make check'.
'make test' triggers the ancient, unmaintained tcg unit tests in
tests/tcg/Makefile which have long since ceased compiling.
Even if someone fixes the TCG tests, it makes little sense to put
them in a 'make test' target, rather they should be 'make check-tcg',
possibly wired up as a dependency of 'make check'.
In the meantime, this patch disarms the 'make test' trap by simply
deleting it so users get an immediate error. This should be enough
for them to remember to type 'make check' instead (or 'make help'
to learn). It also deletes 'make speed' which is another route
into the tcg tests.
Eric Blake [Fri, 17 Nov 2017 19:04:22 +0000 (13:04 -0600)]
iotests: Fix 176 on 32-bit host
The contents of a qcow2 bitmap are rounded up to a size that
matches the number of bits available for the granularity, but
that granularity differs for 32-bit hosts (our default 64k
cluster allows for 2M bitmap coverage per 'long') and 64-bit
hosts (4M bitmap per 'long'). If the image is a multiple of
2M but not 4M, then the number of bytes occupied by the array
of longs in memory differs between architecture, thus
resulting in different SHA256 hashes.
Furthermore (but untested by me), if our computation of the
SHA256 hash is at all endian-dependent because of how we store
data in memory, that's another variable we'd have to account
for (ideally, we specified the bitmap stored in qcow2 as
fixed-endian on disk, because the same qcow2 file must be
usable across any architecture; but that says nothing about
how we represent things in memory). But we already have test
165 to validate that bitmaps are stored correctly on disk,
while this test is merely testing that the bitmap exists.
So for this test, the easiest solution is to filter out the
actual hash value. Broken in commit 4096974e.
Alberto Garcia [Mon, 6 Nov 2017 14:53:45 +0000 (16:53 +0200)]
block: Close a BlockDriverState completely even when bs->drv is NULL
bdrv_close() skips much of its logic when bs->drv is NULL. This is
fine when we're closing a BlockDriverState that has just been created
(because e.g the initialization process failed), but it's not enough
in other cases.
For example, when a valid qcow2 image is found to be corrupted then
QEMU marks it as such in the file header and then sets bs->drv to
NULL in order to make the BlockDriverState unusable. When that BDS is
later closed then many of its data structures are not freed (leaking
their memory) and none of its children are detached. This results in
bdrv_close_all() failing to close all BDSs and making this assertion
fail when QEMU is being shut down:
This patch makes bdrv_close() do the full uninitialization process
in all cases. This fixes the problem with corrupted images and still
works fine with freshly created BDSs.
Kevin Wolf [Mon, 20 Nov 2017 14:41:31 +0000 (15:41 +0100)]
block: Error out on load_vm with active dirty bitmaps
Loading a snapshot invalidates the bitmap. Just marking all blocks dirty
is not a useful response in practice, instead the user needs to be aware
that we switch to a completely different state. If they are okay with
losing the dirty bitmap, they can just explicitly delete it.
Kevin Wolf [Mon, 20 Nov 2017 14:02:48 +0000 (15:02 +0100)]
block: Don't request I/O permission with BDRV_O_NO_IO
'qemu-img info' makes sense even when BLK_PERM_CONSISTENT_READ cannot be
granted because of a block job in a running qemu process. It already
sets BDRV_O_NO_IO to indicate that it doesn't access the guest visible
data at all.
Check the BDRV_O_NO_IO flags in blk_new_open(), so that I/O related
permissions are not unnecessarily requested and 'qemu-img info' can work
even if BLK_PERM_CONSISTENT_READ cannot be granted.
Kevin Wolf [Mon, 20 Nov 2017 13:59:13 +0000 (14:59 +0100)]
block: Don't use BLK_PERM_CONSISTENT_READ for format probing
For format probing, we don't really care whether all of the image
content is consistent. The only thing we're looking at is the image
header, and specifically the magic numbers that are expected to never
change, no matter how inconsistent the guest visible disk content is.
Therefore, don't request BLK_PERM_CONSISTENT_READ. This allows to use
format probing, e.g. in the context of 'qemu-img info', even while the
guest visible data in the image is inconsistent during a running block
job.
Stefan Hajnoczi [Tue, 21 Nov 2017 12:04:35 +0000 (12:04 +0000)]
Use HTTPS for qemu.org and other domains
qemu.org enabled HTTPS in 2017 and it should be used instead of HTTP.
There are also URLs to json.org, openvpn.net, and other domains that
support HTTPS.
This patch updates the qemu.org domains everywhere and also third-party
domains that I have checked.
Stefan Hajnoczi [Tue, 21 Nov 2017 12:04:34 +0000 (12:04 +0000)]
Use qemu.org domain name
The owner of qemu.org has delegated authority to modify DNS records to
the QEMU Project. This has allowed us to use the domain name without
worries about IP address changes or technical issues disrupting service.
The issues described in commit 859389810910f232188675d2f10b15f1aad77660
("Use qemu-project.org domain name") have therefore been mitigated.
This patch switches back to consistently using qemu.org instead of
qemu-project.org in documentation, version.rc, and the Windows installer
script.
The git submodules and SeaBIOS still use qemu-project.org for the time
being. This will be fixed in the QEMU 2.12 release cycle.
Michael Roth [Tue, 7 Nov 2017 20:52:01 +0000 (14:52 -0600)]
scripts/make-release: ship u-boot source as a tarball
The u-boot sources we ship currently cause problems with unpacking on
a case-insensitive filesystem due to path conflicts. This has been
fixed in upstream u-boot via commit 610eec7f, but since it is not
yet included in an official release we implement this approach as a
temporary workaround.
Once we move to a u-boot containing commit 610eec7f we should revert
this patch.
Peter Maydell [Mon, 20 Nov 2017 18:08:28 +0000 (18:08 +0000)]
accel/tcg: Handle atomic accesses to notdirty memory correctly
To do a write to memory that is marked as notdirty, we need
to invalidate any TBs we have cached for that memory, and
update the cpu physical memory dirty flags for VGA and migration.
The slowpath code in notdirty_mem_write() does all this correctly,
but the new atomic handling code in atomic_mmu_lookup() doesn't
do anything at all, it just clears the dirty bit in the TLB.
The effect of this bug is that if the first write to a notdirty
page for which we have cached TBs is by a guest atomic access,
we fail to invalidate the TBs and subsequently will execute
incorrect code. This can be seen by trying to run 'javac' on AArch64.
Use the new notdirty_call_before() and notdirty_call_after()
functions to correctly handle the update to notdirty memory
in the atomic codepath.
Peter Maydell [Mon, 20 Nov 2017 18:08:27 +0000 (18:08 +0000)]
exec.c: Factor out before/after actions for notdirty memory writes
The function notdirty_mem_write() has a sequence of actions
it has to do before and after the actual business of writing
data to host RAM to ensure that dirty flags are correctly
updated and we flush any TCG translations for the region.
We need to do this also in other places that write directly
to host RAM, most notably the TCG atomic helper functions.
Pull out the before and after pieces into their own functions.
We use an API where the prepare function stashes the various
bits of information about the write into a struct for the
complete function to use, because in the calls for the atomic
helpers the place where the complete function will be called
doesn't have the information to hand.
Peter Maydell [Tue, 21 Nov 2017 09:56:05 +0000 (09:56 +0000)]
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20171120' into staging
target-arm queue:
* hw/arm: Silence xlnx-ep108 deprecation warning during tests
* hw/arm/aspeed: Unlock SCU when running kernel
* arm: check regime, not current state, for ATS write PAR format
* nvic: Fix ARMv7M MPU_RBAR reads
* target/arm: Report GICv3 sysregs present in ID registers if needed
* remotes/pmaydell/tags/pull-target-arm-20171120:
hw/arm: Silence xlnx-ep108 deprecation warning during tests
hw/arm/aspeed: Unlock SCU when running kernel
arm: check regime, not current state, for ATS write PAR format
nvic: Fix ARMv7M MPU_RBAR reads
target/arm: Report GICv3 sysregs present in ID registers if needed
ZhiPeng Lu [Fri, 3 Nov 2017 14:54:20 +0000 (22:54 +0800)]
qga: replace GetIfEntry with GetIfEntry2 for interface stats
The data obtained by GetIfEntry is 32 bits, and it may overflow. Thus
using GetIfEntry2 instead of GetIfEntry.
Signed-off-by: ZhiPeng Lu <[email protected]>
*avoid CamelCase variable names
*update field names for MIB_IFROW -> MIB_IF_ROW2
*dynamically probe for GetIfIndex2 to deal with older OSs
*check return value from get_interface_index Signed-off-by: Michael Roth <[email protected]>
* remotes/cohuck/tags/s390x-20171120-v1:
pc-bios/s390-ccw.img: update image
pc-bios/s390-ccw: Fix problem with invalid virtio-scsi LUN when rebooting
s390x/tcg: fix DIAG 308 with > 1 VCPU (MTTCG)
s390x: fix storing CPU status (again)
Peter Maydell [Mon, 20 Nov 2017 18:00:16 +0000 (18:00 +0000)]
Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-2.11-20171120' into staging
ppc patch queue 2017-11-20
Here's the current queue of ppc patches. These 2 patches are both
more complex than I'd ideally like this late in the 2.11 cycle.
However, they do fix important bugs, so I think it's worth it on
balance.
Peter Maydell [Mon, 20 Nov 2017 14:54:16 +0000 (14:54 +0000)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
# gpg: Signature made Mon 20 Nov 2017 03:28:54 GMT
# gpg: using RSA key 0xEF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <[email protected]>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* remotes/jasowang/tags/net-pull-request:
hw/net/vmxnet3: Fix code to work on big endian hosts, too
net: Transmit zero UDP checksum as 0xFFFF
MAINTAINERS: Add missing entry for eepro100 emulation
hw/net/eepro100: Fix endianness problem on big endian hosts
Revert "Add new PCI ID for i82559a"
colo-compare: fix the dangerous assignment
Peter Maydell [Tue, 7 Nov 2017 18:25:18 +0000 (18:25 +0000)]
linux-user: Fix calculation of auxv length
In commit 7c4ee5bcc82e643 we changed the order in which we construct
the AUXV, but forgot to adjust the calculation of the length. The
result is that we set info->auxv_len to a bogus and negative value,
and then later on the code in open_self_auxv() gets confused and
ends up presenting the guest with an empty file.
Since we now have to calculate the auxv length up-front as part
of figuring out how much we're going to put on the stack, set
info->auxv_len then; this allows us to assert that we put the
same number of entries into auxv as we pre-calculated, rather
than merely having a comment saying we need to do that.
Thomas Huth [Thu, 16 Nov 2017 15:29:43 +0000 (16:29 +0100)]
hw/arm: Silence xlnx-ep108 deprecation warning during tests
The new deprecation warning for the xlnx-ep108 machine also pops up
during "make check" which is kind of confusing. Silence it if testing
mode is enabled.
Joel Stanley [Tue, 14 Nov 2017 12:20:18 +0000 (22:50 +1030)]
hw/arm/aspeed: Unlock SCU when running kernel
The ASPEED hardware contains a lock register for the SCU that disables
any writes to the SCU when it is locked. The machine comes up with the
lock enabled, but on all known hardware u-boot will unlock it and leave
it unlocked when loading the kernel.
This means the kernel expects the SCU to be unlocked. When booting from
an emulated ROM the normal u-boot unlock path is executed. Things don't
go well when booting using the -kernel command line, as u-boot does not
run first.
Change behaviour so that when a kernel is passed to the machine, set the
reset value of the SCU to be unlocked.
Peter Maydell [Fri, 3 Nov 2017 14:36:54 +0000 (14:36 +0000)]
arm: check regime, not current state, for ATS write PAR format
In do_ats_write(), rather than using extended_addresses_enabled() to
decide whether the value we get back from get_phys_addr() is a 64-bit
format PAR or a 32-bit one, use arm_s1_regime_using_lpae_format().
This is not really the correct answer, because the PAR format
depends on the AT instruction being used, not just on the
translation regime. However getting this correct requires a
significant refactoring, so that get_phys_addr() returns raw
information about the fault which the caller can then assemble
into a suitable FSR/PAR/syndrome for its purposes, rather than
get_phys_addr() returning a pre-formatted FSR.
However this change at least improves the situation by making
the PAR work correctly for address translation operations done
at AArch64 EL2 on the EL2 translation regime. In particular,
this is necessary for Xen to be able to run in our emulation,
so this seems like a safer interim fix given that we are in freeze.
Peter Maydell [Tue, 7 Nov 2017 15:01:38 +0000 (15:01 +0000)]
target/arm: Report GICv3 sysregs present in ID registers if needed
The CPU ID registers ID_AA64PFR0_EL1, ID_PFR1_EL1 and ID_PFR1
have a field for reporting presence of GICv3 system registers.
We need to report this field correctly in order for Xen to
work as a guest inside QEMU emulation. We mustn't incorrectly
claim the sysregs exist when they don't, though, or Linux will
crash.
Unfortunately the way we've designed the GICv3 emulation in QEMU
puts the system registers as part of the GICv3 device, which
may be created after the CPU proper has been realized. This
means that we don't know at the point when we define the ID
registers what the correct value is. Handle this by switching
them to calling a function at runtime to read the value, where
we can fill in the GIC field appropriately.
Enabling the scsi_req* tracing in QEMU shows that the ccw bios is
trying to execute the REPORT LUNS SCSI command with a LUN != 0, and
this causes the SCSI command to fail.
Looks like we neither clear the BSS of the s390-ccw bios during reboot,
nor do we explicitly set the default_scsi_device.lun value to 0, so
this variable can contain random values from the OS after the reboot.
By setting this variable explicitly to 0, the problem is fixed and
the reboots always succeed.
Thomas Huth [Tue, 14 Nov 2017 11:20:24 +0000 (12:20 +0100)]
hw/net/vmxnet3: Fix code to work on big endian hosts, too
Since commit ab06ec43577177a442e8 we test the vmxnet3 device in the
pxe-tester, too (when running "make check SPEED=slow"). This now
revealed that the code is not working there if the host is a big
endian machine (for example ppc64 or s390x) - "make check SPEED=slow"
is now failing on such hosts.
The vmxnet3 code lacks endianness conversions in a couple of places.
Interestingly, the bitfields in the structs in vmxnet3.h already tried to
take care of the *bit* endianness of the C compilers - but the code missed
to change the *byte* endianness when reading or writing the corresponding
structs. So the bitfields are now wrapped into unions which allow to change
the byte endianness during runtime with the non-bitfield member of the union.
With these changes, "make check SPEED=slow" now properly works on big endian
hosts, too.
projects / qemu.git / log