memory: Make memory_region_readd_subregion() properly handle mapped aliases
memory_region_readd_subregion() wants to readd a region by first
removing it and then readding it. For readding, it doesn't use one of
the memory_region_add_*() variants, which is why fail to re-increment the
mr->mapped_via_alias counters, resulting in the
assert(alias->mapped_via_alias >= 0) in memory_region_del_subregion()
triggering the next time we call memory_region_readd_subregion().
Fix it by using memory_region_add_subregion_common() for readding the
region.
Peter Maydell [Mon, 28 Mar 2022 09:16:33 +0000 (10:16 +0100)]
Merge tag 'pull-ppc-20220326' of https://github.com/legoater/qemu into staging
ppc-7.0 queue:
* target/ppc: fix helper_xvmadd* argument order
# gpg: Signature made Sat 26 Mar 2022 12:57:54 GMT
# gpg: using RSA key A0F66548F04895EBFE6B0B6051A343C7CFFBECA1
# gpg: Good signature from "Cédric Le Goater <[email protected]>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: A0F6 6548 F048 95EB FE6B 0B60 51A3 43C7 CFFB ECA1
* tag 'pull-ppc-20220326' of https://github.com/legoater/qemu:
target/ppc: fix helper_xvmadd* argument order
Matheus Ferst [Sat, 26 Mar 2022 12:52:37 +0000 (13:52 +0100)]
target/ppc: fix helper_xvmadd* argument order
When the xsmadd* insns were moved to decodetree, the helper arguments
were reordered to better match the PowerISA description. The same macro
is used to declare xvmadd* helpers, but the translation macro of these
insns was not changed accordingly.
* tag 'pull-target-arm-20220325' of https://git.linaro.org/people/pmaydell/qemu-arm:
hw/intc/arm_gicv3_its: Add missing newlines to process_mapc() logging
target/arm: Fix sve_ld1_z and sve_st1_z vs MMIO
Peter Maydell [Fri, 25 Mar 2022 14:41:06 +0000 (14:41 +0000)]
hw/intc/arm_gicv3_its: Add missing newlines to process_mapc() logging
In commit 84d43d2e82da we rearranged the logging of errors in
process_mapc(), and inadvertently dropped the trailing newlines
from the log messages. Restore them. The same commit also
attempted to switch the ICID printing to hex (which is how we
print ICIDs elsewhere) but only did half the job, adding the
0x prefix but leaving the format string at %d; correct to %x.
Fixes: 84d43d2e82da ("hw/intc/arm_gicv3_its: In MAPC with V=0, don't check rdbase field") Signed-off-by: Peter Maydell <[email protected]> Reviewed-by: Richard Henderson <[email protected]> Reviewed-by: Philippe Mathieu-Daudé <[email protected]>
* tag 'for-upstream' of https://gitlab.com/bonzini/qemu:
build: disable fcf-protection on -march=486 -m16
target/i386: properly reset TSC on reset
target/i386: tcg: high bits SSE cmp operation must be ignored
configure: remove dead int128 test
KVM: x86: workaround invalid CPUID[0xD,9] info on some AMD processors
i386: Set MCG_STATUS_RIPV bit for mce SRAR error
target/i386/kvm: Free xsave_buf when destroying vCPU
Peter Maydell [Thu, 24 Mar 2022 20:08:39 +0000 (20:08 +0000)]
Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging
Pull request
Philippe found cases where the 0x%d format string was used, leading to
misleading output. The patches look harmless and could save people time, so I
think it's worth including them in 7.0.
# gpg: Signature made Thu 24 Mar 2022 17:15:20 GMT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <[email protected]>" [full]
# gpg: aka "Stefan Hajnoczi <[email protected]>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* tag 'block-pull-request' of https://gitlab.com/stefanha/qemu:
hw: Fix misleading hexadecimal format
block: Fix misleading hexadecimal format
Some of the roms build with -march=i486 -m16 which is incompatible
with -fcf-protection. That in turn is can be set by default, for
example in Ubuntu [1].
That causes:
cc1: error: ‘-fcf-protection’ is not compatible with this target
This won't work on -march=i486 -m16 and no matter if set or not we can
override it to "none" if the option is known to the compiler to be
able to build reliably.
Paolo Bonzini [Thu, 24 Mar 2022 08:21:41 +0000 (09:21 +0100)]
target/i386: properly reset TSC on reset
Some versions of Windows hang on reboot if their TSC value is greater
than 2^54. The calibration of the Hyper-V reference time overflows
and fails; as a result the processors' clock sources are out of sync.
The issue is that the TSC _should_ be reset to 0 on CPU reset and
QEMU tries to do that. However, KVM special cases writing 0 to the
TSC and thinks that QEMU is trying to hot-plug a CPU, which is
correct the first time through but not later. Thwart this valiant
effort and reset the TSC to 1 instead, but only if the CPU has been
run once.
For this to work, env->tsc has to be moved to the part of CPUArchState
that is not zeroed at the beginning of x86_cpu_reset.
Paolo Bonzini [Thu, 24 Mar 2022 08:08:39 +0000 (09:08 +0100)]
target/i386: tcg: high bits SSE cmp operation must be ignored
High bits in the immediate operand of SSE comparisons are ignored, they
do not result in an undefined opcode exception. This is mentioned
explicitly in the Intel documentation.
Reported-by: [email protected] Closes: https://gitlab.com/qemu-project/qemu/-/issues/184 Signed-off-by: Paolo Bonzini <[email protected]>
linux-user/arm: Implement __kernel_cmpxchg64 with host atomics
If CONFIG_ATOMIC64, we can use a host cmpxchg and provide
atomicity across processes; otherwise we have no choice but
to continue using start/end_exclusive.
Paolo Bonzini [Wed, 23 Mar 2022 11:33:25 +0000 (12:33 +0100)]
KVM: x86: workaround invalid CPUID[0xD,9] info on some AMD processors
Some AMD processors expose the PKRU extended save state even if they do not have
the related PKU feature in CPUID. Worse, when they do they report a size of
64, whereas the expected size of the PKRU extended save state is 8, therefore
the esa->size == eax assertion does not hold.
The state is already ignored by KVM_GET_SUPPORTED_CPUID because it
was not enabled in the host XCR0. However, QEMU kvm_cpu_xsave_init()
runs before QEMU invokes arch_prctl() to enable dynamically-enabled
save states such as XTILEDATA, and KVM_GET_SUPPORTED_CPUID hides save
states that have yet to be enabled. Therefore, kvm_cpu_xsave_init()
needs to consult the host CPUID instead of KVM_GET_SUPPORTED_CPUID,
and dies with an assertion failure.
When setting up the ExtSaveArea array to match the host, ignore features that
KVM does not report as supported. This will cause QEMU to skip the incorrect
CPUID leaf instead of tripping the assertion.
Peter Maydell [Wed, 23 Mar 2022 11:45:35 +0000 (11:45 +0000)]
Merge tag 'pull-fixes-for-7.0-230322-1' of https://github.com/stsquad/qemu into staging
Various fixes for 7.0
- make clean also cleans tcg tests
- fix rounding error in i386 fildl[l]
- more clean-ups to gitdm/mailmap metadata
- apply some organisation to docs/devel
- clean-up semihosting argv handling
- add custom runner for aarch32
- remove old qemu_logo.pdf
- document QEMU_PLUGIN env var
# gpg: Signature made Wed 23 Mar 2022 10:41:03 GMT
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <[email protected]>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* tag 'pull-fixes-for-7.0-230322-1' of https://github.com/stsquad/qemu:
docs/tcg-plugins: document QEMU_PLUGIN behaviour
docs: remove qemu_logo.pdf
gitlab: include new aarch32 job in custom-runners
semihosting: clean up handling of expanded argv
docs/devel: try and impose some organisation
mailmap/gitdm: more fixes for bad tags and authors
target/i386: force maximum rounding precision for fildl[l]
tests/Makefile.include: Let "make clean" remove the TCG tests, too
luofei [Thu, 20 Jan 2022 08:46:34 +0000 (03:46 -0500)]
i386: Set MCG_STATUS_RIPV bit for mce SRAR error
In the physical machine environment, when a SRAR error occurs,
the IA32_MCG_STATUS RIPV bit is set, but qemu does not set this
bit. When qemu injects an SRAR error into virtual machine, the
virtual machine kernel just call do_machine_check() to kill the
current task, but not call memory_failure() to isolate the faulty
page, which will cause the faulty page to be allocated and used
repeatedly. If used by the virtual machine kernel, it will cause
the virtual machine to crash
target/i386/kvm: Free xsave_buf when destroying vCPU
Fix vCPU hot-unplug related leak reported by Valgrind:
==132362== 4,096 bytes in 1 blocks are definitely lost in loss record 8,440 of 8,549
==132362== at 0x4C3B15F: memalign (vg_replace_malloc.c:1265)
==132362== by 0x4C3B288: posix_memalign (vg_replace_malloc.c:1429)
==132362== by 0xB41195: qemu_try_memalign (memalign.c:53)
==132362== by 0xB41204: qemu_memalign (memalign.c:73)
==132362== by 0x7131CB: kvm_init_xsave (kvm.c:1601)
==132362== by 0x7148ED: kvm_arch_init_vcpu (kvm.c:2031)
==132362== by 0x91D224: kvm_init_vcpu (kvm-all.c:516)
==132362== by 0x9242C9: kvm_vcpu_thread_fn (kvm-accel-ops.c:40)
==132362== by 0xB2EB26: qemu_thread_start (qemu-thread-posix.c:556)
==132362== by 0x7EB2159: start_thread (in /usr/lib64/libpthread-2.28.so)
==132362== by 0x9D45DD2: clone (in /usr/lib64/libc-2.28.so)
QEMU plugins can be loaded via command line arguments or via
the QEMU_PLUGIN environment variable. Currently, only the first method
is documented. Let's document QEMU_PLUGIN.
As drive-by cleanup, this patch fixes the path to the plugins
in the same section of the documentation.
Alex Bennée [Wed, 16 Mar 2022 16:05:22 +0000 (16:05 +0000)]
docs: remove qemu_logo.pdf
As far as I can tell nothing in the source tree uses this file. The
original comment mentions texi2pdf and I think that was all culled
when we switched to our rst documentation pipeline.
Alex Bennée [Tue, 15 Mar 2022 11:46:12 +0000 (11:46 +0000)]
semihosting: clean up handling of expanded argv
Another cleanup patch tripped over the fact we weren't being careful
in our casting. Fix the casts, allow for a non-const and switch from
g_realloc to g_renew.
The whole semihosting argument handling could do with some tests
though.
Alex Bennée [Wed, 9 Mar 2022 13:52:15 +0000 (13:52 +0000)]
docs/devel: try and impose some organisation
We have a growing set of developer docs but the index is currently in
order of when stuff was added. Try and make things a bit easier to
find my adding sub indexes and organising into themes.
Alex Bennée [Fri, 11 Mar 2022 13:41:53 +0000 (13:41 +0000)]
mailmap/gitdm: more fixes for bad tags and authors
I was running some historical tags for the last 10 years and got the
following warnings:
git log --use-mailmap --numstat --since "June 2010" | ~/src/gitdm.git/gitdm -n -l 5 [email protected] is an author name, probably not what you want
bad utf-8 ('utf-8' codec can't decode byte 0xe4 in position 552: invalid continuation byte) in patchm skipping
bad utf-8 ('utf-8' codec can't decode byte 0xe4 in position 342: invalid continuation byte) in patchm skipping [email protected] is an author name, probably not what you want
Oops...funky email nicta.com.au
bad utf-8 ('utf-8' codec can't decode byte 0xe9 in position 232: invalid continuation byte) in patchm skipping
Oops...funky email andreas.faerber
Grabbing changesets...done
Processed 76422 csets from 1902 developers
The following fixes try and alleviate that although I still get a
warning for Aaron which I think is from 9743cd5736.
Thomas Huth [Tue, 1 Mar 2022 08:59:00 +0000 (09:59 +0100)]
tests/Makefile.include: Let "make clean" remove the TCG tests, too
"make clean" should clear all binaries that have been built, but so
far it left the TCG tests still in place. Let's make sure that they
are now removed, too.
* tag 'fixes-pull-request' of gitlab.com:marcandre.lureau/qemu: (21 commits)
qapi: remove needless include
Remove trailing ; after G_DEFINE_AUTO macro
tests: remove needless include
error: use GLib to remember the program name
qga: remove bswap.h include
qapi: remove needless include
meson: fix CONFIG_ATOMIC128 check
meson: move int128 checks from configure
qapi: remove needless include
util: remove the net/net.h dependency
util: remove needless includes
scripts/modinfo-collect: remove unused/dead code
Move HOST_LONG_BITS to compiler.h
Simplify HOST_LONG_BITS
compiler.h: replace QEMU_SENTINEL with G_GNUC_NULL_TERMINATED
compiler.h: replace QEMU_WARN_UNUSED_RESULT with G_GNUC_WARN_UNUSED_RESULT
Replace GCC_FMT_ATTR with G_GNUC_PRINTF
Drop qemu_foo() socket API wrapper
m68k/nios2-semi: fix gettimeofday() result check
vl: typo fix in a comment
...
Alex Bennée [Mon, 21 Mar 2022 16:11:51 +0000 (16:11 +0000)]
gitlab: disable accelerated zlib for s390x
There appears to be a bug in the s390 hardware-accelerated version of
zlib distributed with Ubuntu 20.04, which makes our test
/i386/migration/multifd/tcp/zlib hit an assertion perhaps one time in
10. Fortunately zlib provides an escape hatch where we can disable the
hardware-acceleration entirely by setting the environment variable
DFLTCC to 0. Do this on all our CI which runs on s390 hosts, both our
custom gitlab runner and also the Travis hosts.
Peter Maydell [Tue, 22 Mar 2022 12:44:11 +0000 (12:44 +0000)]
Merge tag 'pull-block-2022-03-22' of https://gitlab.com/hreitz/qemu into staging
Block patches for 7.0-rc1:
- iotest fixes:
- Fix some iotests for riscv targets
- Use GNU sed in more places where required
- Meson-related fixes (i.e. to print errors when they occur)
- Have qemu-img calls (from Python tests) generally raise nicely
formattable exceptions on errors
- Fix iotest 207
- Allow RBD images to be growable by writing zeroes past the end of
file, fixing qcow2 on rbd
# gpg: Signature made Tue 22 Mar 2022 11:51:10 GMT
# gpg: using RSA key CB62D7A0EE3829E45F004D34A1FA40D098019CDF
# gpg: issuer "[email protected]"
# gpg: Good signature from "Hanna Reitz <[email protected]>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: CB62 D7A0 EE38 29E4 5F00 4D34 A1FA 40D0 9801 9CDF
* tag 'pull-block-2022-03-22' of https://gitlab.com/hreitz/qemu: (25 commits)
iotests/207: Filter host fingerprint
iotests.py: Filters for VM.run_job()
iotests: make qemu_img_log and img_info_log raise on error
iotests: remove qemu_img_pipe_and_status()
iotests: replace qemu_img_log('create', ...) calls
iotests: use qemu_img() in has_working_luks()
iotests: remove remaining calls to qemu_img_pipe()
iotests/149: Remove qemu_img_pipe() call
iotests: replace unchecked calls to qemu_img_pipe()
iotests: change supports_quorum to use qemu_img
iotests: add qemu_img_map() function
iotests/remove-bitmap-from-backing: use qemu_img_info()
iotests: add qemu_img_info()
iotests: use qemu_img_json() when applicable
iotests: add qemu_img_json()
iotests: fortify compare_images() against crashes
iotests: make qemu_img raise on non-zero rc by default
iotests: Remove explicit checks for qemu_img() == 0
python/utils: add VerboseProcessError
python/utils: add add_visual_margin() text decoration utility
...
Unblocked signals are never delivered, because we
didn't record the new mask for process_pending_signals.
Handle this with the same mechanism as sigsuspend.
linux-user: Properly handle sigset arg to epoll_pwait
Unblocked signals are never delivered, because we
didn't record the new mask for process_pending_signals.
Handle this with the same mechanism as sigsuspend.
Unblocked signals are never delivered, because we
didn't record the new mask for process_pending_signals.
Handle this with the same mechanism as sigsuspend.
linux-user/alpha: Fix sigsuspend for big-endian hosts
On alpha, the sigset argument for sigsuspend is in a register.
When we drop that into memory that happens in host-endianness,
but target_to_host_old_sigset will treat it as target-endianness.
WANG Xuerui [Sun, 20 Mar 2022 05:22:59 +0000 (13:22 +0800)]
linux-user: Fix syscall parameter handling for MIPS n32
The MIPS n32 ABI is basically n64 with the address space (i.e. pointer
width) shrinked to 32 bits. Meanwhile the current code treats it as
o32-like based on TARGET_ABI_BITS, which causes problems with n32
syscalls utilizing 64-bit offsets, like pread64, affecting most (if not
all) recently built n32 binaries.
This partially solves issue #909 ("qemu-mipsn32(el) user mode emulator
fails to execute any recently built n32 binaries"); with this change
applied, the built qemu-mipsn32el is able to progress beyond the
pread64, and finish _dl_start_user for the "getting ld.so load libc.so"
case. The program later dies with SIGBUS, though, due to _dl_start_user
not maintaining stack alignment after removing ld.so itself from argv,
and qemu-user starting to enforce alignment recently, but that is
orthogonal to the issue here; the more common case of chrooting is
working, verified with my own-built Gentoo n32 sysroot. (Depending on
the exact ISA used, one may have to explicitly specify QEMU_CPU, which
is the case for my chroot.)
The current testing code isn't correct and matching QEMU usage:
testfile.c: In function 'main':
testfile.c:5:11: error: incorrect number of arguments to function '__atomic_load'
5 | y = __atomic_load(&x, 0);
| ^~~~~~~~~~~~~
testfile.c:6:7: error: argument 2 of '__atomic_store' must be a pointer type
6 | __atomic_store(&x, y, 0);
| ^~~~~~~~~~~~~~
testfile.c:7:7: error: argument 3 of '__atomic_compare_exchange' must be a pointer type
7 | __atomic_compare_exchange(&x, &y, x, 0, 0, 0);
| ^~~~~~~~~~~~~~~~~~~~~~~~~
Replace the test with common atomics test for u64 and u128 that matches
better QEMU needs.
The socket API wrappers were initially introduced in commit 00aa0040 ("Wrap recv to avoid warnings"), but made redundant with
commit a2d96af4 ("osdep: add wrappers for socket functions") which fixes
the win32 declarations and thus removed the earlier warnings.
Peter Maydell [Tue, 22 Mar 2022 10:17:57 +0000 (10:17 +0000)]
Merge tag 'pull-nbd-2022-03-21' of https://repo.or.cz/qemu/ericb into staging
nbd patches for 2022-03-21
- Rao Lei: fix nbd client hang on server death
- Vladimir Sementsov-Ogievskiy: email address update
- Eric Blake: qemu-nbd documentation tweak
# gpg: Signature made Mon 21 Mar 2022 21:35:50 GMT
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <[email protected]>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <[email protected]>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A
* tag 'pull-nbd-2022-03-21' of https://repo.or.cz/qemu/ericb:
block/nbd.c: Fixed IO request coroutine not being wakeup when kill NBD server
docs: Consistent typography for options of qemu-nbd
MAINTAINERS: change Vladimir's email address
Hanna Reitz [Fri, 18 Mar 2022 12:53:04 +0000 (13:53 +0100)]
iotests/207: Filter host fingerprint
Commit e3296cc796aeaf319f3ed4e064ec309baf5e4da4 made the ssh block
driver's error message for fingerprint mismatches more verbose, so it
now prints the actual host key fingerprint and the key type.
iotest 207 tests such errors, but was not amended to filter that
fingerprint (which is host-specific), so do it now. Filter the key
type, too, because I guess this too can differ depending on the host
configuration.
Hanna Reitz [Fri, 18 Mar 2022 12:53:03 +0000 (13:53 +0100)]
iotests.py: Filters for VM.run_job()
Allow filters for VM.run_job(), and pass the filters given to
VM.blockdev_create() to it.
(Use this opportunity to annotate VM.run_job()'s parameter types;
unfortunately, for the filter, I could not come up with anything better
than Callable[[Any], Any] that would pass mypy's scrutiny.)
At one point, a plain string is logged, so the filters passed to it must
work fine with plain strings. The only filters passed to it at this
point are the ones from VM.blockdev_create(), which are
filter_qmp_test_files() (by default) and 207's filter_hash(). Both
cannot handle plain strings yet, but we can make them by amending
filter_qmp() to treat them as plain values with a None key.
iotests 242 and 266 need to use check=False for their negative tests.
iotests 206, 210, 211, 212, 213, 237, 274 and 302 continue working
normally.
As of this commit, all calls to QEMU_IMG made from iotests enforce a
return code of zero by default unless explicitly disabled or suppressed
by passing check=False or with an exception handler.
John Snow [Mon, 21 Mar 2022 20:16:17 +0000 (16:16 -0400)]
iotests: remove qemu_img_pipe_and_status()
With the exceptional 'create' calls removed in the prior commit, change
qemu_img_log() and img_info_log() to call qemu_img() directly
instead.
For now, allow these calls to qemu-img to return non-zero on the basis
that any unusual output will be logged anyway. The very next commit
begins to enforce a successful exit code by default even for the logged
functions.
qemu_img_log() calls into qemu_img_pipe(), which always removes output
for 'create' commands on success anyway. Replace all of these calls to
the simpler qemu_img_create(...) which doesn't log, but raises a
detailed exception object on failure instead.
Blank lines are removed from output files where appropriate.
John Snow [Mon, 21 Mar 2022 20:16:15 +0000 (16:16 -0400)]
iotests: use qemu_img() in has_working_luks()
Admittedly a mostly lateral move, but qemu_img() is essentially the
replacement for qemu_img_pipe_and_status(). It will give slightly better
diagnostics on crash.
John Snow [Mon, 21 Mar 2022 20:16:14 +0000 (16:16 -0400)]
iotests: remove remaining calls to qemu_img_pipe()
As part of moving all python iotest invocations of qemu-img onto a
single qemu_img() implementation, remove a few lingering uses of
qemu_img_pipe() from outside of iotests.py itself.
Several cases here rely on the knowledge that qemu_img_pipe() suppresses
*all* output on a successful case when the command being issued is
'create'.
065: This call's output is inspected, but it appears as if it's expected
to succeed. Replace this call with the checked qemu_img() variant
instead to get better diagnostics if/when qemu-img itself fails.
237: "create" call output isn't actually logged. Use qemu_img_create()
instead, which checks the return code. Remove the empty lines from
the test output.
296: Two calls;
-create: Expected to succeed. Like other create calls, the output
isn't actually logged. Switch to a checked variant
(qemu_img_create) instead. The output for this test is
a mixture of both test styles, so actually replace the
blank line for readability.
-amend: This is expected to fail. Log the output.
After this patch, the only uses of qemu_img_pipe are internal to
iotests.py and will be removed in subsequent patches.
John Snow [Mon, 21 Mar 2022 20:16:13 +0000 (16:16 -0400)]
iotests/149: Remove qemu_img_pipe() call
qemu_img_pipe calls blank their output when the command being run is a
'create' call and the command succeeds. Thus, the normative output for
this command in iotest 149 is to print a blank line. We can remove the
logging from this invocation and use a checked invocation, but we still
need to inspect the actual output to see if we want to retroactively
skip the test due to missing cipher support.
John Snow [Mon, 21 Mar 2022 20:16:12 +0000 (16:16 -0400)]
iotests: replace unchecked calls to qemu_img_pipe()
qemu_img_pipe() discards the return code from qemu-img in favor of
returning just its output. Some tests using this function don't save,
log, or check the output either, though, which is unsafe.
Replace all of these calls with a checked version.
Tests affected are 194, 202, 203, 234, 262, and 303.
John Snow [Mon, 21 Mar 2022 20:16:11 +0000 (16:16 -0400)]
iotests: change supports_quorum to use qemu_img
Similar to other recent changes: use the qemu_img() invocation that
supports throwing loud, nasty exceptions when it fails for surprising
reasons.
(Why would "--help" ever fail? I don't know, but eliminating *all* calls
to qemu-img that do not go through qemu_img() is my goal, so
qemu_img_pipe() has to be removed.)
John Snow [Mon, 21 Mar 2022 20:16:10 +0000 (16:16 -0400)]
iotests: add qemu_img_map() function
Add a qemu_img_map() function by analogy with qemu_img_measure(),
qemu_img_check(), and qemu_img_info() that all return JSON information.
Replace calls to qemu_img_pipe('map', '--output=json', ...) with this
new function, which provides better diagnostic information on failure.
Note: The output for iotest 211 changes, because logging JSON after it
was deserialized by Python behaves a little differently than logging the
raw JSON document string itself.
(iotests.log() sorts the keys for Python 3.6 support.)
John Snow [Mon, 21 Mar 2022 20:16:08 +0000 (16:16 -0400)]
iotests: add qemu_img_info()
Add qemu_img_info() by analogy with qemu_img_measure() and
qemu_img_check(). Modify image_size() to use this function instead to
take advantage of the better diagnostic information on failure provided
(ultimately) by qemu_img().
John Snow [Mon, 21 Mar 2022 20:16:06 +0000 (16:16 -0400)]
iotests: add qemu_img_json()
qemu_img_json() is a new helper built on top of qemu_img() that tries to
pull a valid JSON document out of the stdout stream.
In the event that the return code is negative (the program crashed), or
the code is greater than zero and did not produce valid JSON output, the
VerboseProcessError raised by qemu_img() is re-raised.
In the event that the return code is zero but we can't parse valid JSON,
allow the JSON deserialization error to be raised.
John Snow [Mon, 21 Mar 2022 20:16:05 +0000 (16:16 -0400)]
iotests: fortify compare_images() against crashes
Fortify compare_images() to be more discerning about the status codes it
receives. If qemu_img() returns an exit code that implies it didn't
actually perform the comparison, treat that as an exceptional
circumstance and force the caller to be aware of the peril.
If a negative test is desired (perhaps to test how qemu_img compare
behaves on malformed images, for instance), it is still possible to
catch the exception in the test and deal with that circumstance
manually.
John Snow [Mon, 21 Mar 2022 20:16:04 +0000 (16:16 -0400)]
iotests: make qemu_img raise on non-zero rc by default
re-write qemu_img() as a function that will by default raise a
VerboseProcessException (extended from CalledProcessException) on
non-zero return codes. This will produce a stack trace that will show
the command line arguments and return code from the failed process run.
Users that want something more flexible (there appears to be only one)
can use check=False and manage the return themselves. However, when the
return code is negative, the Exception will be raised no matter what.
This is done under the belief that there's no legitimate reason, even in
negative tests, to see a crash from qemu-img.
John Snow [Mon, 21 Mar 2022 20:16:03 +0000 (16:16 -0400)]
iotests: Remove explicit checks for qemu_img() == 0
qemu_img() returning zero ought to be the rule, not the
exception. Remove all explicit checks against the condition in
preparation for making non-zero returns an Exception.
John Snow [Mon, 21 Mar 2022 20:16:02 +0000 (16:16 -0400)]
python/utils: add VerboseProcessError
This adds an Exception that extends the Python stdlib
subprocess.CalledProcessError.
The difference is that the str() method of this exception also adds the
stdout/stderr logs. In effect, if this exception goes unhandled, Python
will print the output in a visually distinct wrapper to the terminal so
that it's easy to spot in a sea of traceback information.
John Snow [Mon, 21 Mar 2022 20:16:01 +0000 (16:16 -0400)]
python/utils: add add_visual_margin() text decoration utility
>>> print(add_visual_margin(msg, width=72, name="Commit Message"))
┏━ Commit Message ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
┃ add_visual_margin() takes a chunk of text and wraps it in a visual
┃ container that force-wraps to a specified width. An optional title
┃ label may be given, and any of the individual glyphs used to draw the
┃ box may be replaced or specified as well.
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Thomas Huth [Wed, 23 Feb 2022 09:58:16 +0000 (10:58 +0100)]
tests/qemu-iotests/testrunner: Supply a test plan in TAP mode
Quoting the TAP specification: "The plan tells how many tests will be
run [...]. It’s a check that the test file hasn’t stopped prematurely."
That's a good idea of course, so let's support that in the iotest
testrunner, too.
Thomas Huth [Thu, 10 Mar 2022 07:50:48 +0000 (08:50 +0100)]
tests: Do not treat the iotests as separate meson test target anymore
If there is a failing iotest, the output is currently not logged to
the console anymore. To get this working again, we need to run the
meson test runner with "--print-errorlogs" (and without "--verbose"
due to a current meson bug that will be fixed here:
https://github.com/mesonbuild/meson/commit/c3f145ca2b9f5.patch ).
We could update the "meson test" call in tests/Makefile.include,
but actually it's nicer and easier if we simply do not treat the
iotests as separate test target anymore and integrate them along
with the other test suites. This has the disadvantage of not getting
the detailed progress indication there anymore, but since that was
only working right in single-threaded "make -j1" mode anyway, it's
not a huge loss right now.
Thomas Huth [Wed, 9 Mar 2022 10:16:26 +0000 (11:16 +0100)]
tests/qemu-iotests: Use GNU sed in two more spots where it is necessary
These two spots have been missed in commit 9086c7639822 ("Rework the
checks and spots using GNU sed") - they need GNU sed, too, since they
are using the "+" address form.
Commit d24f80234b ("block/rbd: increase dynamically the image size")
added a workaround to support growing images (eg. qcow2), resizing
the image before write operations that exceed the current size.
We recently added support for write zeroes and without the
workaround we can have problems with qcow2.
So let's move the resize into qemu_rbd_start_co() and do it when
the command is RBD_AIO_WRITE or RBD_AIO_WRITE_ZEROES.
laokz [Tue, 8 Mar 2022 04:33:39 +0000 (12:33 +0800)]
tests: add (riscv virt) machine mapping to testenv
Some qemu-iotests(040 etc) use PCI disk to do test. Without the
mapping, RISC-V flavor use spike as default machine which has no
PCI bus, causing test failure.
* tag 'for-upstream' of https://gitlab.com/bonzini/qemu:
hw/i386/amd_iommu: Fix maybe-uninitialized error with GCC 12
target/i386: kvm: do not access uninitialized variable on older kernels
Rao Lei [Wed, 9 Mar 2022 07:48:44 +0000 (15:48 +0800)]
block/nbd.c: Fixed IO request coroutine not being wakeup when kill NBD server
During the IO stress test, the IO request coroutine has a probability that is
can't be awakened when the NBD server is killed.
The GDB stack is as follows:
(gdb) bt
0 0x00007f2ff990cbf6 in __ppoll (fds=0x55575de85000, nfds=1, timeout=<optimized out>, sigmask=0x0) at ../sysdeps/unix/sysv/linux/ppoll.c:44
1 0x000055575c302e7c in qemu_poll_ns (fds=0x55575de85000, nfds=1, timeout=599999603140) at ../util/qemu-timer.c:348
2 0x000055575c2d3c34 in fdmon_poll_wait (ctx=0x55575dc480f0, ready_list=0x7ffd9dd1dae0, timeout=599999603140) at ../util/fdmon-poll.c:80
3 0x000055575c2d350d in aio_poll (ctx=0x55575dc480f0, blocking=true) at ../util/aio-posix.c:655
4 0x000055575c16eabd in bdrv_do_drained_begin(bs=0x55575dee7fe0, recursive=false, parent=0x0, ignore_bds_parents=false, poll=true)at ../block/io.c:474
5 0x000055575c16eba6 in bdrv_drained_begin (bs=0x55575dee7fe0) at ../block/io.c:480
6 0x000055575c1aff33 in quorum_del_child (bs=0x55575dee7fe0, child=0x55575dcea690, errp=0x7ffd9dd1dd08) at ../block/quorum.c:1130
7 0x000055575c14239b in bdrv_del_child (parent_bs=0x55575dee7fe0, child=0x55575dcea690, errp=0x7ffd9dd1dd08) at ../block.c:7705
8 0x000055575c12da28 in qmp_x_blockdev_change(parent=0x55575df404c0 "colo-disk0", has_child=true, child=0x55575de867f0 "children.1", has_node=false, no de=0x0, errp=0x7ffd9dd1dd08) at ../blockdev.c:3676
9 0x000055575c258435 in qmp_marshal_x_blockdev_change (args=0x7f2fec008190, ret=0x7f2ff7b0bd98, errp=0x7f2ff7b0bd90) at qapi/qapi-commands-block-core.c :1675
10 0x000055575c2c6201 in do_qmp_dispatch_bh (opaque=0x7f2ff7b0be30) at ../qapi/qmp-dispatch.c:129
11 0x000055575c2ebb1c in aio_bh_call (bh=0x55575dc429c0) at ../util/async.c:141
12 0x000055575c2ebc2a in aio_bh_poll (ctx=0x55575dc480f0) at ../util/async.c:169
13 0x000055575c2d2d96 in aio_dispatch (ctx=0x55575dc480f0) at ../util/aio-posix.c:415
14 0x000055575c2ec07f in aio_ctx_dispatch (source=0x55575dc480f0, callback=0x0, user_data=0x0) at ../util/async.c:311
15 0x00007f2ff9e7cfbd in g_main_context_dispatch () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
16 0x000055575c2fd581 in glib_pollfds_poll () at ../util/main-loop.c:232
17 0x000055575c2fd5ff in os_host_main_loop_wait (timeout=0) at ../util/main-loop.c:255
18 0x000055575c2fd710 in main_loop_wait (nonblocking=0) at ../util/main-loop.c:531
19 0x000055575bfa7588 in qemu_main_loop () at ../softmmu/runstate.c:726
20 0x000055575bbee57a in main (argc=60, argv=0x7ffd9dd1e0e8, envp=0x7ffd9dd1e2d0) at ../softmmu/main.c:50
(gdb) qemu coroutine 0x55575e16aac0
0 0x000055575c2ee7dc in qemu_coroutine_switch (from_=0x55575e16aac0, to_=0x7f2ff830fba0, action=COROUTINE_YIELD) at ../util/coroutine-ucontext.c:302
1 0x000055575c2fe2a9 in qemu_coroutine_yield () at ../util/qemu-coroutine.c:195
2 0x000055575c2fe93c in qemu_co_queue_wait_impl (queue=0x55575dc46170, lock=0x7f2b32ad9850) at ../util/qemu-coroutine-lock.c:56
3 0x000055575c17ddfb in nbd_co_send_request (bs=0x55575ebfaf20, request=0x7f2b32ad9920, qiov=0x55575dfc15d8) at ../block/nbd.c:478
4 0x000055575c17f931 in nbd_co_request (bs=0x55575ebfaf20, request=0x7f2b32ad9920, write_qiov=0x55575dfc15d8) at ../block/nbd.c:1182
5 0x000055575c17fe14 in nbd_client_co_pwritev (bs=0x55575ebfaf20, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, flags=0) at ../block/nbd.c:1284
6 0x000055575c170d25 in bdrv_driver_pwritev (bs=0x55575ebfaf20, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, qiov_offset=0, flags=0)
at ../block/io.c:1264
7 0x000055575c1733b4 in bdrv_aligned_pwritev
(child=0x55575dff6890, req=0x7f2b32ad9ad0, offset=403487858688, bytes=4538368, align=1, qiov=0x55575dfc15d8, qiov_offset=0, flags=0) at ../block/io.c:2126
8 0x000055575c173c67 in bdrv_co_pwritev_part (child=0x55575dff6890, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, qiov_offset=0, flags=0)
at ../block/io.c:2314
9 0x000055575c17391b in bdrv_co_pwritev (child=0x55575dff6890, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, flags=0) at ../block/io.c:2233
10 0x000055575c1ee506 in replication_co_writev (bs=0x55575e9824f0, sector_num=788062224, remaining_sectors=8864, qiov=0x55575dfc15d8, flags=0)
at ../block/replication.c:270
11 0x000055575c170eed in bdrv_driver_pwritev (bs=0x55575e9824f0, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, qiov_offset=0, flags=0)
at ../block/io.c:1297
12 0x000055575c1733b4 in bdrv_aligned_pwritev
(child=0x55575dcea690, req=0x7f2b32ad9e00, offset=403487858688, bytes=4538368, align=512, qiov=0x55575dfc15d8, qiov_offset=0, flags=0)
at ../block/io.c:2126
13 0x000055575c173c67 in bdrv_co_pwritev_part (child=0x55575dcea690, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, qiov_offset=0, flags=0)
at ../block/io.c:2314
14 0x000055575c17391b in bdrv_co_pwritev (child=0x55575dcea690, offset=403487858688, bytes=4538368, qiov=0x55575dfc15d8, flags=0) at ../block/io.c:2233
15 0x000055575c1aeffa in write_quorum_entry (opaque=0x7f2fddaf8c50) at ../block/quorum.c:699
16 0x000055575c2ee4db in coroutine_trampoline (i0=1578543808, i1=21847) at ../util/coroutine-ucontext.c:173
17 0x00007f2ff9855660 in __start_context () at ../sysdeps/unix/sysv/linux/x86_64/__start_context.S:91
When we do failover in COLO mode, QEMU will hang while it is waiting for
the in-flight IO. From the call trace, we can see the IO request coroutine
has yielded in nbd_co_send_request(). When we kill the NBD server, it will never
be wake up. Actually, when we do IO stress test, it will have a lot of
requests in free_sema queue. When the NBD server is killed, current
MAX_NBD_REQUESTS finishes with errors but they wake up at most
MAX_NBD_REQEUSTS from the queue. So, let's move qemu_co_queue_next out
to fix this issue.
* tag 'pull-misc-2022-03-21' of git://repo.or.cz/qemu/armbru:
Use g_new() & friends where that makes obvious sense
9pfs: Use g_new() & friends where that makes obvious sense
scripts/coccinelle: New use-g_new-etc.cocci
block-qdict: Fix -Werror=maybe-uninitialized build failure
Peter Maydell [Mon, 21 Mar 2022 15:27:13 +0000 (15:27 +0000)]
Merge tag 'block-pull-request' of https://gitlab.com/stefanha/qemu into staging
Pull request
Bug fixes for 7.0.
# gpg: Signature made Thu 17 Mar 2022 16:42:22 GMT
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <[email protected]>" [full]
# gpg: aka "Stefan Hajnoczi <[email protected]>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* tag 'block-pull-request' of https://gitlab.com/stefanha/qemu:
aio-posix: fix spurious ->poll_ready() callbacks in main loop
aio-posix: fix build failure io_uring 2.2
Paolo Bonzini [Mon, 21 Mar 2022 14:30:18 +0000 (15:30 +0100)]
hw/i386/amd_iommu: Fix maybe-uninitialized error with GCC 12
Be more explicit that the loop must roll at least once. Avoids the
following warning:
FAILED: libqemu-x86_64-softmmu.fa.p/hw_i386_amd_iommu.c.o
In function 'pte_get_page_mask',
inlined from 'amdvi_page_walk' at hw/i386/amd_iommu.c:945:25,
inlined from 'amdvi_do_translate' at hw/i386/amd_iommu.c:989:5,
inlined from 'amdvi_translate' at hw/i386/amd_iommu.c:1038:5:
hw/i386/amd_iommu.c:877:38: error: 'oldlevel' may be used uninitialized [-Werror=maybe-uninitialized]
877 | return ~((1UL << ((oldlevel * 9) + 3)) - 1);
| ~~~~~~~~~~~~~~~~^~~~
hw/i386/amd_iommu.c: In function 'amdvi_translate':
hw/i386/amd_iommu.c:906:41: note: 'oldlevel' was declared here
906 | unsigned level, present, pte_perms, oldlevel;
| ^~~~~~~~
cc1: all warnings being treated as errors
Use g_new() & friends where that makes obvious sense
g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer,
for two reasons. One, it catches multiplication overflowing size_t.
Two, it returns T * rather than void *, which lets the compiler catch
more type errors.
This commit only touches allocations with size arguments of the form
sizeof(T).
9pfs: Use g_new() & friends where that makes obvious sense
g_new(T, n) is neater than g_malloc(sizeof(T) * n). It's also safer,
for two reasons. One, it catches multiplication overflowing size_t.
Two, it returns T * rather than void *, which lets the compiler catch
more type errors.
This commit only touches allocations with size arguments of the form
sizeof(T).
../hw/9pfs/9p.c: In function ‘qid_path_fullmap’:
../hw/9pfs/9p.c:855:13: error: assignment to ‘QpfEntry *’ from incompatible pointer type ‘QppEntry *’ [-Werror=incompatible-pointer-types]
855 | val = g_new0(QppEntry, 1);
| ^
Harmless, because QppEntry is larger than QpfEntry. Manually fixed to
allocate a QpfEntry instead.
Building QEMU on Fedora 37 (Rawhide Prerelease) ppc64le failed with the
following error:
$ ../configure --prefix=/usr/local/qemu-disabletcg --target-list=ppc-softmmu,ppc64-softmmu --disable-tcg --disable-linux-user
...
$ make -j$(nproc)
...
In file included from /root/qemu/include/qapi/qmp/qdict.h:16,
from /root/qemu/include/block/qdict.h:13,
from ../qobject/block-qdict.c:11:
/root/qemu/include/qapi/qmp/qobject.h: In function ‘qdict_array_split’:
/root/qemu/include/qapi/qmp/qobject.h:49:17: error: ‘subqdict’ may be used uninitialized [-Werror=maybe-uninitialized]
49 | typeof(obj) _obj = (obj); \
| ^~~~
../qobject/block-qdict.c:227:16: note: ‘subqdict’ declared here
227 | QDict *subqdict;
| ^~~~~~~~
cc1: all warnings being treated as errors
Fix build failure by expanding the ternary operation.
Tested with `make check-unit` (the check-block-qdict test passed).
projects / qemu.git / log