Bin Meng [Fri, 1 May 2020 15:50:54 +0000 (08:50 -0700)]
riscv: Suppress the error report for QEMU testing with riscv_find_firmware()
We only ship plain binary bios images in the QEMU source. With Spike
machine that uses ELF images as the default bios, running QEMU test
will complain hence let's suppress the error report for QEMU testing.
Peter Maydell [Tue, 2 Jun 2020 17:16:38 +0000 (18:16 +0100)]
Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20200602' into staging
Vector rotate support
Signal handling support for NetBSD arm/aarch64
# gpg: Signature made Tue 02 Jun 2020 17:43:05 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "[email protected]"
# gpg: Good signature from "Richard Henderson <[email protected]>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F
* remotes/rth/tags/pull-tcg-20200602:
accel/tcg: Provide a NetBSD specific aarch64 cpu_signal_handler
accel/tcg: Adjust cpu_signal_handler for NetBSD/arm
tcg: Improve move ops in liveness_pass_2
target/s390x: Use tcg_gen_gvec_rotl{i,s,v}
target/ppc: Use tcg_gen_gvec_rotlv
tcg/ppc: Implement INDEX_op_rot[lr]v_vec
tcg/aarch64: Implement INDEX_op_rotl{i,v}_vec
tcg/i386: Implement INDEX_op_rotl{i,s,v}_vec
tcg: Implement gvec support for rotate by scalar
tcg: Remove expansion to shift by vector from do_shifts
tcg: Implement gvec support for rotate by vector
tcg: Implement gvec support for rotate by immediate
If the output of the move is dead, then the last use is in
the store. If we propagate the input to the store, then we
can remove the move opcode entirely.
For immediate rotate , we can implement this in two instructions,
using SLI. For variable rotate, the oddness of aarch64 right-shift-
as-negative-left-shift means a backend-specific expansion works best.
No host backend support yet, but the interfaces for rotls
are in place. Only implement left-rotate for now, as the
only known use of vector rotate by scalar is s390x, so any
right-rotate would be unused and untestable.
tcg: Remove expansion to shift by vector from do_shifts
We do not reflect this expansion in tcg_can_emit_vecop_list,
so it is unused and unusable. However, we actually perform
the same expansion in do_gvec_shifts, so it is also unneeded.
No host backend support yet, but the interfaces for rotlv
and rotrv are in place.
Reviewed-by: Alex Bennée <[email protected]> Signed-off-by: Richard Henderson <[email protected]>
---
v3: Drop the generic expansion from rot to shift; we can do better
for each backend, and then this code becomes unused.
tcg: Implement gvec support for rotate by immediate
No host backend support yet, but the interfaces for rotli
are in place. Canonicalize immediate rotate to the left,
based on a survey of architectures, but provide both left
and right shift interfaces to the translators.
Peter Maydell [Mon, 1 Jun 2020 20:34:47 +0000 (21:34 +0100)]
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20200601a' into staging
Migration/virtio/hmp pull 2020-06-01
A mixed pull with:
- RDMA migration fix (CID 1428762)
- HMP qom-get addition and qom-set cleanup
- a virtiofsd fix
- COLO fixes
Signed-off-by: Dr. David Alan Gilbert <[email protected]>
# gpg: Signature made Mon 01 Jun 2020 19:37:15 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <[email protected]>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20200601a:
migration/migration.c: Fix hang in ram_save_host_page
migration/colo.c: Move colo_notify_compares_event to the right place
migration/colo.c: Relaunch failover even if there was an error
migration/colo.c: Flush ram cache only after receiving device state
migration/colo.c: Use cpu_synchronize_all_states()
migration/colo.c: Use event instead of semaphore
migration/vmstate: Remove unnecessary MemoryRegion forward declaration
virtiofsd: remove symlink fallbacks
hmp: Simplify qom-set
hmp: Implement qom-get HMP command
migration/rdma: cleanup rdma context before g_free to avoid memleaks
migration/rdma: fix potential nullptr access in rdma_start_incoming_migration
Peter Maydell [Mon, 1 Jun 2020 18:19:18 +0000 (19:19 +0100)]
Merge remote-tracking branch 'remotes/bkoppelmann2/tags/pull-tricore-20200601' into staging
Remove ctx->env ptr, add TriCore gdb stub
# gpg: Signature made Mon 01 Jun 2020 16:06:35 BST
# gpg: using RSA key 6E636A7E83F2DD0CFA6E6E370AD2C6396B69CA14
# gpg: issuer "[email protected]"
# gpg: Good signature from "Bastian Koppelmann <[email protected]>" [full]
# Primary key fingerprint: 6E63 6A7E 83F2 DD0C FA6E 6E37 0AD2 C639 6B69 CA14
* remotes/bkoppelmann2/tags/pull-tricore-20200601:
target/tricore: Implement gdbstub
target/tricore: Implement tricore_cpu_get_phys_page_debug
target/tricore: Raise EXCP_DEBUG in gen_goto_tb() for singlestep
target/tricore: Move translate feature check to ctx
target/tricore: Don't save pc in generate_qemu_excp
Lukas Straub [Wed, 20 May 2020 20:42:32 +0000 (22:42 +0200)]
migration/migration.c: Fix hang in ram_save_host_page
migration_rate_limit will erroneously ratelimit a shutdown socket,
which causes the migration thread to hang in ram_save_host_page
if the socket is shutdown.
Fix this by explicitly testing if the socket has errors or was
shutdown in migration_rate_limit.
Lukas Straub [Mon, 11 May 2020 11:11:01 +0000 (13:11 +0200)]
migration/colo.c: Move colo_notify_compares_event to the right place
If the secondary has to failover during checkpointing, it still is
in the old state (i.e. different state than primary). Thus we can't
expose the primary state until after the checkpoint is sent.
This fixes sporadic connection reset of client connections during
failover.
Lukas Straub [Mon, 11 May 2020 11:10:55 +0000 (13:10 +0200)]
migration/colo.c: Relaunch failover even if there was an error
If vmstate_loading is true, secondary_vm_do_failover will set failover
status to FAILOVER_STATUS_RELAUNCH and return success without initiating
failover. However, if there is an error during the vmstate_loading
section, failover isn't relaunched. Instead we then wait for
failover on colo_incoming_sem.
Fix this by relaunching failover even if there was an error. Also,
to make this work properly, set vmstate_loading to false when
returning during the vmstate_loading section.
Lukas Straub [Mon, 11 May 2020 11:10:48 +0000 (13:10 +0200)]
migration/colo.c: Use cpu_synchronize_all_states()
cpu_synchronize_all_pre_loadvm() marks all vcpus as dirty, so the
registers are loaded from CPUState before we continue running
the vm. However if we failover during checkpoint, CPUState is not
initialized and the registers are loaded with garbage. This causes
guest hangs and crashes.
Fix this by using cpu_synchronize_all_states(), which initializes
CPUState from the current cpu registers additionally to marking
the vcpus as dirty.
Lukas Straub [Mon, 11 May 2020 11:10:44 +0000 (13:10 +0200)]
migration/colo.c: Use event instead of semaphore
If multiple packets miscompare in a short timeframe, the semaphore
value will be increased multiple times. This causes multiple
checkpoints even if one would be sufficient.
Fix this by using a event instead of a semaphore for triggering
checkpoints. Now, checkpoint requests will be ignored until the
checkpoint event is sent to colo-compare (which releases the
miscompared packets).
Benchmark results (iperf3):
Client-to-server tcp:
without patch: ~66 Mbit/s
with patch: ~61 Mbit/s
Server-to-client tcp:
without patch: ~702 Kbit/s
with patch: ~16 Mbit/s
Miklos Szeredi [Thu, 14 May 2020 14:07:36 +0000 (16:07 +0200)]
virtiofsd: remove symlink fallbacks
Path lookup in the kernel has special rules for looking up magic symlinks
under /proc. If a filesystem operation is instructed to follow symlinks
(e.g. via AT_SYMLINK_FOLLOW or lack of AT_SYMLINK_NOFOLLOW), and the final
component is such a proc symlink, then the target of the magic symlink is
used for the operation, even if the target itself is a symlink. I.e. path
lookup is always terminated after following a final magic symlink.
I was erronously assuming that in the above case the target symlink would
also be followed, and so workarounds were added for a couple of operations
to handle the symlink case. Since the symlink can be handled simply by
following the proc symlink, these workardouds are not needed.
Also remove the "norace" option, which disabled the workarounds.
Commit bdfd66788349 ("virtiofsd: Fix xattr operations") already dealt with
the same issue for xattr operations.
This started off as Andreas Färber's implementation from
March 2015, but after feedback from Paolo and Markus it morphed into
using the json output which handles structs reasonably.
Use with qom-list to find the members of an object.
Peter Maydell [Mon, 1 Jun 2020 12:43:59 +0000 (13:43 +0100)]
Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-june-01-2020' into staging
MIPS queue for June 1st, 2020
# gpg: Signature made Mon 01 Jun 2020 12:29:25 BST
# gpg: using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <[email protected]>" [full]
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01 DD75 D497 2A89 67F7 5A65
* remotes/amarkovic/tags/mips-queue-june-01-2020:
hw/mips: fuloong2e: Set preferred page size to 16KB
target/mips: Support variable page size
target/mips: Add more CP0 register for save/restore
hw/mips: Add CPU IRQ3 delivery for KVM
configure: Add KVM target support for MIPS64
tests/Makefile: Fix description of "make check"
Traditionally, MIPS use 4KB page size, but Loongson prefer 16KB page
size in system emulator. So, let's define TARGET_PAGE_BITS_VARY and
TARGET_PAGE_BITS_MIN to support variable page size.
Huacai Chen [Sun, 3 May 2020 10:20:17 +0000 (18:20 +0800)]
hw/mips: Add CPU IRQ3 delivery for KVM
Currently, KVM/MIPS only deliver I/O interrupt via IP2, this patch add
IP3 delivery as well, because Loongson-3 based machine use both IRQ2
(CPU's IP2) and IRQ3 (CPU's IP3).
Peter Maydell [Sun, 31 May 2020 20:49:07 +0000 (21:49 +0100)]
Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200531' into staging
Python queue:
* migration acceptance test fix
* introduce pylintrc & flake8 config
* various cleanups (Python3, style)
* vm-test can set QEMU_LOCAL=1 to use locally built binaries
* refactored BootLinuxBase & LinuxKernelTest acceptance classes
Pavel Dovgalyuk [Fri, 29 May 2020 07:04:39 +0000 (10:04 +0300)]
tests/acceptance: allow console interaction with specific VMs
Console interaction in avocado scripts was possible only with single
default VM.
This patch modifies the function parameters to allow passing a specific
VM as a parameter to interact with it.
tests/acceptance/migration.py: Wait for both sides
When the source finishes migration the destination will still be
receiving the data sent by the source, so it might not have quite
finished yet, so won't quite have reached 'completed'.
This lead to occasional asserts in the next few checks.
After the source has finished, check the destination as well.
(We can't just switch to checking the destination, because it doesn't
give a status until it has started receiving the migration).
John Snow [Thu, 14 May 2020 05:53:44 +0000 (01:53 -0400)]
python/qemu: Adjust traceback typing
mypy considers it incorrect to use `bool` to statically return false,
because it will assume that it could conceivably return True, and gives
different analysis in that case. Use a None return to achieve the same
effect, but make mypy happy.
Note: Pylint considers function signatures as code that might trip the
duplicate-code checker. I'd rather not disable this as it does not
trigger often in practice, so I'm disabling it as a one-off and filed a
change request; see https://github.com/PyCQA/pylint/issues/3619
John Snow [Thu, 14 May 2020 05:53:43 +0000 (01:53 -0400)]
python/qemu: fix socket.makefile() typing
Note:
A bug in typeshed (https://github.com/python/typeshed/issues/3977)
misinterprets the type of makefile(). Work around this by explicitly
stating that we are opening a text-mode file.
John Snow [Thu, 28 May 2020 22:21:29 +0000 (18:21 -0400)]
python/qemu: delint; add flake8 config
Mostly, ignore the "no bare except" rule, because flake8 is not
contextual and cannot determine if we re-raise. Pylint can, though, so
always prefer pylint for that.
John Snow [Thu, 28 May 2020 22:21:28 +0000 (18:21 -0400)]
python/qemu: delint and add pylintrc
Bring our these files up to speed with pylint 2.5.0.
Add a pylintrc file to formalize which pylint subset
we are targeting.
The similarity ignore is there to suppress similarity
reports across imports, which for typing constants,
are going to trigger this report erroneously.
John Snow [Thu, 14 May 2020 05:53:52 +0000 (01:53 -0400)]
python/qemu/machine: remove logging configuration
Python 3.5 and above do not print a warning when logging is not
configured. As a library, it's best practice to leave logging
configuration to the client executable.
John Snow [Thu, 14 May 2020 03:52:30 +0000 (23:52 -0400)]
python: remove more instances of sys.version_info
We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.
John Snow [Thu, 28 May 2020 22:21:26 +0000 (18:21 -0400)]
scripts/qmp: Fix shebang and imports
There's more wrong with these scripts; They are in various stages of
disrepair. That's beyond the scope of this current patchset.
This just mechanically corrects the imports and the shebangs, as part of
ensuring that the python/qemu/lib refactoring didn't break anything
needlessly.
Peter Maydell [Fri, 29 May 2020 18:25:54 +0000 (19:25 +0100)]
Merge remote-tracking branch 'remotes/ericb/tags/pull-bitmaps-2020-05-26-v3' into staging
bitmaps patches for 2020-05-26
- fix non-blockdev migration of bitmaps when mirror job is in use
- add bitmap sizing to 'qemu-img measure'
- add 'qemu-img convert --bitmaps'
# gpg: Signature made Thu 28 May 2020 19:16:47 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <[email protected]>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <[email protected]>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A
* remotes/ericb/tags/pull-bitmaps-2020-05-26-v3:
iotests: Add test 291 to for qemu-img bitmap coverage
qemu-img: Add convert --bitmaps option
qemu-img: Factor out code for merging bitmaps
qcow2: Expose bitmaps' size during measure
iotests: Fix test 178
migration: forbid bitmap migration by generated node-name
migration: add_bitmaps_to_list: check disk name once
iotests: 194: test also migration of dirty bitmap
migration: fix bitmaps pre-blockdev migration with mirror job
block/dirty-bitmap: add bdrv_has_named_bitmaps helper
migration: refactor init_dirty_bitmap_migration
Peter Maydell [Fri, 29 May 2020 16:41:45 +0000 (17:41 +0100)]
Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1' into staging
Testing and one plugin fix:
- support alternates for genisoimage to test/vm
- add clang++ to clang tests
- fix record/replay smoke test
- enable more softfloat tests
- better detection of hung gdb
- upgrade aarch64 tcg test x-compile to gcc-10
- fix plugin cpu_index clash vs threads
# gpg: Signature made Wed 27 May 2020 14:29:20 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <[email protected]>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-tcg-plugins-270520-1:
tests/tcg: add new threadcount test
linux-user: properly "unrealize" vCPU object
cpus-common: ensure auto-assigned cpu_indexes don't clash
tests/docker: use a gcc-10 based image for arm64 tests
tests/docker: add debian11 base image
tests/tcg: better detect confused gdb which can't connect
tests/fp: split and audit the conversion tests
tests/fp: enable extf80_le_quite tests
tests/tcg: fix invocation of the memory record/replay tests
travis.yml: Use clang++ in the Clang tests
tests/vm: pass --genisoimage to basevm script
configure: add alternate binary for genisoimage
Eric Blake [Thu, 21 May 2020 19:21:36 +0000 (14:21 -0500)]
qemu-img: Add convert --bitmaps option
Make it easier to copy all the persistent bitmaps of (the top layer
of) a source image along with its guest-visible contents, by adding a
boolean flag for use with qemu-img convert. This is basically
shorthand, as the same effect could be accomplished with a series of
'qemu-img bitmap --add' and 'qemu-img bitmap --merge -b source'
commands, or by their corresponding QMP commands.
Note that this command will fail in the same scenarios where 'qemu-img
measure' omits a 'bitmaps size:' line, namely, when either the source
or the destination lacks persistent bitmap support altogether.
See also https://bugzilla.redhat.com/show_bug.cgi?id=1779893
While touching this, clean up a couple coding issues spotted in the
same function: an extra blank line, and merging back-to-back 'if
(!skip_create)' blocks.
Eric Blake [Thu, 21 May 2020 19:21:35 +0000 (14:21 -0500)]
qemu-img: Factor out code for merging bitmaps
The next patch will add another client that wants to merge dirty
bitmaps; it will be easier to refactor the code to construct the QAPI
struct correctly into a helper function.
Eric Blake [Thu, 21 May 2020 19:21:34 +0000 (14:21 -0500)]
qcow2: Expose bitmaps' size during measure
It's useful to know how much space can be occupied by qcow2 persistent
bitmaps, even though such metadata is unrelated to the guest-visible
data. Report this value as an additional QMP field, present when
measuring an existing image and output format that both support
bitmaps. Update iotest 178 and 190 to updated output, as well as new
coverage in 190 demonstrating non-zero values made possible with the
recently-added qemu-img bitmap command (see 3b51ab4b).
The new 'bitmaps size:' field is displayed automatically as part of
'qemu-img measure' any time it is present in QMP (that is, any time
both the source image being measured and destination format support
bitmaps, even if the measurement is 0 because there are no bitmaps
present). If the field is absent, it means that no bitmaps can be
copied (source, destination, or both lack bitmaps, including when
measuring based on size rather than on a source image). This behavior
is compatible with an upcoming patch adding 'qemu-img convert
--bitmaps': that command will fail in the same situations where this
patch omits the field.
The addition of a new field demonstrates why we should always
zero-initialize qapi C structs; while the qcow2 driver still fully
populates all fields, the raw and crypto drivers had to be tweaked to
avoid uninitialized data.
Consideration was also given towards having a 'qemu-img measure
--bitmaps' which errors out when bitmaps are not possible, and
otherwise sums the bitmaps into the existing allocation totals rather
than displaying as a separate field, as a potential convenience
factor. But this was ultimately decided to be more complexity than
necessary when the QMP interface was sufficient enough with bitmaps
remaining a separate field.
Eric Blake [Thu, 21 May 2020 19:21:33 +0000 (14:21 -0500)]
iotests: Fix test 178
A recent change to qemu-img changed expected error message output, but
178 takes long enough to execute that it does not get run by 'make
check' or './check -g quick'.
migration: fix bitmaps pre-blockdev migration with mirror job
Important thing for bitmap migration is to select destination block
node to obtain the migrated bitmap.
Prepatch, on source we use bdrv_get_device_or_node_name() to identify
the node, and on target we do bdrv_lookup_bs.
bdrv_get_device_or_node_name() returns blk name only for direct
children of blk. So, bitmaps of direct children of blks are migrated by
blk name and others - by node name.
Old libvirt is unprepared to bitmap migration by node-name,
node-names are mostly auto-generated. So actually only migration by blk
name works for it.
Newer libvirt will use new interface (which will be added soon) to
specify node-mapping for bitmaps migration explicitly. Still, let's
improve the current behavior a bit.
Now, consider classic libvirt migrations assisted by mirror block job:
mirror block job inserts filter, so our source is not a direct child of
blk, and bitmaps are migrated by node-names. And this just doesn't work
with auto-generated node names.
Let's fix it by using blk-name even if some implicit filters are
inserted.
Note2: we, of course, can't skip filters and use blk name to migrate
bitmaps in filtered node by blk name for this blk if these filters have
named bitmaps which should be migrated.
* remotes/kraxel/tags/vga-20200528-pull-request: (21 commits)
sm501: Remove obsolete changelog and todo comment
sm501: Optimize small overlapping blits
sm501: Replace hand written implementation with pixman where possible
sm501: Clean up local variables in sm501_2d_operation
sm501: Use BIT(x) macro to shorten constant
sm501: Shorten long variable names in sm501_2d_operation
sm501: Convert printf + abort to qemu_log_mask
hw/display/pxa2xx_lcd: Replace printf() call by qemu_log_mask()
hw/display/omap_dss: Replace fprintf() call by qemu_log_mask(LOG_UNIMP)
hw/display/exynos4210_fimd: Use qemu_log_mask(GUEST_ERROR)
hw/display/vmware_vga: Let the PCI device own its I/O MemoryRegion
hw/display/vmware_vga: Replace printf() calls by qemu_log_mask(ERROR)
hw/display/xlnx_dp: Replace disabled DPRINTF() by error_report()
hw/display/dpcd: Convert debug printf()s to trace events
hw/display/dpcd: Fix memory region size
hw/display/cirrus_vga: Convert debug printf() to trace event
hw/display/cirrus_vga: Use qemu_log_mask(ERROR) instead of debug printf
hw/display/cirrus_vga: Use qemu_log_mask(UNIMP) instead of debug printf
hw/display/cirrus_vga: Convert debug printf() to trace event
hw/display/cg3: Convert debug printf()s to trace events
...
Peter Maydell [Thu, 28 May 2020 15:18:06 +0000 (16:18 +0100)]
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-05-28' into staging
Fixes and improvements for the gitlab-CI:
- Build with other images instead of the broken Debian containers
- Fix building with the latest version of Clang (at least wrt. to
the gitlab-CI pipeline)
- Add Philippe, Alex and Wainer to the Gitlab-CI section in MAINTAINERS
* remotes/huth-gitlab/tags/pull-request-2020-05-28:
gitlab-ci: Determine the number of jobs dynamically
gitlab-ci: Do not use the standard container images from gitlab
gitlab-ci: Move edk2 and opensbi YAML files to .gitlab-ci.d folder
GitLab CI: avoid calling before_scripts on unintended jobs
gitlab-ci: Remove flex/bison packages
MAINTAINERS: Add Philippe, Alex and Wainer to the Gitlab-CI section
linux-user: limit check to HOST_LONG_BITS < TARGET_ABI_BITS
BALATON Zoltan [Thu, 21 May 2020 19:39:44 +0000 (21:39 +0200)]
sm501: Optimize small overlapping blits
AmigaOS tends to do a lot of small blits (even 1 pixel). Avoid malloc
overhead by keeping around a buffer for this and only alloc when
blitting larger areas.
BALATON Zoltan [Thu, 21 May 2020 19:39:44 +0000 (21:39 +0200)]
sm501: Convert printf + abort to qemu_log_mask
Some places already use qemu_log_mask() to log unimplemented features
or errors but some others have printf() then abort(). Convert these to
qemu_log_mask() and avoid aborting to prevent guests to easily cause
denial of service.
projects / qemu.git / log