* remotes/bonzini/tags/for-upstream: (29 commits)
hw/i386/pc: Extract the x86 generic fw_cfg code
hw/i386/pc: Rename pc_build_feature_control() as generic fw_cfg_build_*
hw/i386/pc: Let pc_build_feature_control() take a MachineState argument
hw/i386/pc: Let pc_build_feature_control() take a FWCfgState argument
hw/i386/pc: Rename pc_build_smbios() as generic fw_cfg_build_smbios()
hw/i386/pc: Let pc_build_smbios() take a generic MachineState argument
hw/i386/pc: Let pc_build_smbios() take a FWCfgState argument
hw/i386/pc: Replace PCMachineState argument with MachineState in fw_cfg_arch_create
hw/i386/pc: Pass the CPUArchIdList array by argument
hw/i386/pc: Pass the apic_id_limit value by argument
hw/i386/pc: Pass the boot_cpus value by argument
hw/i386/pc: Rename bochs_bios_init as more generic fw_cfg_arch_create
hw/i386/pc: Use address_space_memory in place
hw/i386/pc: Extract e820 memory layout code
hw/i386/pc: Use e820_get_num_entries() to access e820_entries
cpus: Fix throttling during vm_stop
qemu-thread: Add qemu_cond_timedwait
memory: inline and optimize devend_memop
memory: fetch pmem size in get_file_size()
elf-ops.h: fix int overflow in load_elf()
...
Extract all the functions that are not PC-machine specific into
the (arch-specific) fw_cfg.c file. This will allow other X86-machine
to reuse these functions.
Paolo Bonzini [Mon, 16 Sep 2019 10:42:42 +0000 (12:42 +0200)]
hw/i386/pc: Replace PCMachineState argument with MachineState in fw_cfg_arch_create
In the previous commit we removed the last access to PCMachineState.
Replace it with a generic MachineState argument and use it to retrieve
the CPUArchIdList.
hw/i386/pc: Rename bochs_bios_init as more generic fw_cfg_arch_create
The bochs_bios_init() function is not restricted to the Bochs
BIOS and is useful to other BIOS.
Since it is not specific to the PC machine, and can be reused
by other machines of the X86 architecture, rename it as
fw_cfg_arch_create().
Throttling thread sleeps in VCPU thread. For high throttle percentage
this sleep is more than 10ms. E.g. for 60% - 15ms, for 99% - 990ms.
vm_stop() kicks all VCPUs and waits for them. It's called at the end of
migration and because of the long sleep the migration downtime might be
more than 100ms even for downtime-limit 1ms.
Use qemu_cond_timedwait for high percentage to wake up during vm_stop.
The new function is needed to implement conditional sleep for CPU
throttling. It's possible to reuse qemu_sem_timedwait, but it's more
difficult than just add qemu_cond_timedwait.
Also moved compute_abs_deadline function up the code to reuse it in
qemu_cond_timedwait_impl win32.
Peter Maydell [Mon, 16 Sep 2019 14:25:55 +0000 (15:25 +0100)]
Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2019-09-16' into staging
Block patches:
- Fix for block jobs when used with I/O threads
- Fix for a corruption when using qcow2's LUKS encryption mode
- cURL fix
- check-block.sh cleanups (for make check)
- Refactoring
* remotes/maxreitz/tags/pull-block-2019-09-16:
qemu-iotests: Add test for bz #1745922
block/qcow2: refactor encryption code
block/qcow2: Fix corruption introduced by commit 8ac0f15f335
blockjob: update nodes head while removing all bdrv
curl: Check curl_multi_add_handle()'s return code
curl: Handle success in multi_check_completion
curl: Report only ready sockets
curl: Pass CURLSocket to curl_multi_do()
curl: Check completion in curl_multi_do()
curl: Keep *socket until the end of curl_sock_cb()
curl: Keep pointer to the CURLState in CURLSocket
tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache
tests/Makefile: Do not print the name of the check-block.sh shell script
tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text
block: Remove unused masks
block: Use QEMU_IS_ALIGNED
* Change the qcow2_co_{encrypt|decrypt} to just receive full host and
guest offsets and use this function directly instead of calling
do_perform_cow_encrypt (which is removed by that patch).
* Adjust qcow2_co_encdec to take full host and guest offsets as well.
* Document the qcow2_co_{encrypt|decrypt} arguments
to prevent the bug fixed in former commit from hopefully
happening again.
The corruption happens when we do a write that
* writes to two or more unallocated clusters at once
* doesn't fully cover the first sector
* doesn't fully cover the last sector
* uses luks encryption
In this case, when allocating the new clusters we COW both areas
prior to the write and after the write, and we encrypt them.
The above mentioned commit accidentally made it so we encrypt the
second COW area using the physical cluster offset of the first area.
The problem is that offset_in_cluster in do_perform_cow_encrypt
can be larger that the cluster size, thus cluster_offset
will no longer point to the start of the cluster at which encrypted
area starts.
Next patch in this series will refactor the code to avoid all these
assumptions.
In the bugreport that was triggered by rebasing a luks image to new,
zero filled base, which lot of such writes, and causes some files
with zero areas to contain garbage there instead.
But as described above it can happen elsewhere as well
blockjob: update nodes head while removing all bdrv
block_job_remove_all_bdrv() iterates through job->nodes, calling
bdrv_root_unref_child() for each entry. The call to the latter may
reach child_job_[can_]set_aio_ctx(), which will also attempt to
traverse job->nodes, potentially finding entries that where freed
on previous iterations.
To avoid this situation, update job->nodes head on each iteration to
ensure that already freed entries are no longer linked to the list.
Max Reitz [Tue, 10 Sep 2019 12:41:35 +0000 (14:41 +0200)]
curl: Handle success in multi_check_completion
Background: As of cURL 7.59.0, it verifies that several functions are
not called from within a callback. Among these functions is
curl_multi_add_handle().
curl_read_cb() is a callback from cURL and not a coroutine. Waking up
acb->co will lead to entering it then and there, which means the current
request will settle and the caller (if it runs in the same coroutine)
may then issue the next request. In such a case, we will enter
curl_setup_preadv() effectively from within curl_read_cb().
Calling curl_multi_add_handle() will then fail and the new request will
not be processed.
Fix this by not letting curl_read_cb() wake up acb->co. Instead, leave
the whole business of settling the AIOCB objects to
curl_multi_check_completion() (which is called from our timer callback
and our FD handler, so not from any cURL callbacks).
Max Reitz [Tue, 10 Sep 2019 12:41:34 +0000 (14:41 +0200)]
curl: Report only ready sockets
Instead of reporting all sockets to cURL, only report the one that has
caused curl_multi_do_locked() to be called. This lets us get rid of the
QLIST_FOREACH_SAFE() list, which was actually wrong: SAFE foreaches are
only safe when the current element is removed in each iteration. If it
possible for the list to be concurrently modified, we cannot guarantee
that only the current element will be removed. Therefore, we must not
use QLIST_FOREACH_SAFE() here.
Max Reitz [Tue, 10 Sep 2019 12:41:33 +0000 (14:41 +0200)]
curl: Pass CURLSocket to curl_multi_do()
curl_multi_do_locked() currently marks all sockets as ready. That is
not only inefficient, but in fact unsafe (the loop is). A follow-up
patch will change that, but to do so, curl_multi_do_locked() needs to
know exactly which socket is ready; and that is accomplished by this
patch here.
Max Reitz [Tue, 10 Sep 2019 12:41:32 +0000 (14:41 +0200)]
curl: Check completion in curl_multi_do()
While it is more likely that transfers complete after some file
descriptor has data ready to read, we probably should not rely on it.
Better be safe than sorry and call curl_multi_check_completion() in
curl_multi_do(), too, just like it is done in curl_multi_read().
With this change, curl_multi_do() and curl_multi_read() are actually the
same, so drop curl_multi_read() and use curl_multi_do() as the sole FD
handler.
Max Reitz [Tue, 10 Sep 2019 12:41:30 +0000 (14:41 +0200)]
curl: Keep pointer to the CURLState in CURLSocket
A follow-up patch will make curl_multi_do() and curl_multi_read() take a
CURLSocket instead of the CURLState. They still need the latter,
though, so add a pointer to it to the former.
tests/qemu-iotests: Fix qemu-io related output in 026.out.nocache
qemu-io now prefixes its error and warnings with "qemu-io:". 36b9986b08787019e fixed a lot of iotests output but forget about
026.out.nocache. Fix it too.
Thomas Huth [Fri, 6 Sep 2019 11:35:34 +0000 (13:35 +0200)]
tests/Makefile: Do not print the name of the check-block.sh shell script
The check script is already printing out which iotest is currently
running, so printing out the name of the check-block.sh shell script
looks superfluous here.
Thomas Huth [Fri, 6 Sep 2019 11:39:20 +0000 (13:39 +0200)]
tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text
When running "make check -j8" or something similar, the iotests are
running in parallel with the other tests. So when they are printing
out "Passed all xx tests" or a similar status message at the end,
it might not be quite clear that this message belongs to the iotests,
since the output might be mixed with the other tests. Thus change the
word "tests" here to "iotests" instead to avoid confusion.
Peter Maydell [Mon, 16 Sep 2019 12:21:28 +0000 (13:21 +0100)]
Merge remote-tracking branch 'remotes/rth/tags/pull-hppa-20190915' into staging
Two temp live across branch fixes.
# gpg: Signature made Sun 15 Sep 2019 14:48:06 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "[email protected]"
# gpg: Good signature from "Richard Henderson <[email protected]>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F
* remotes/rth/tags/pull-hppa-20190915:
target/hppa: prevent trashing of temporary in do_depw_sar()
target/hppa: prevent trashing of temporary in trans_mtctl()
Paolo Bonzini [Thu, 12 Sep 2019 14:02:42 +0000 (16:02 +0200)]
memory: inline and optimize devend_memop
devend_memop can rely on the fact that the result is always either
0 or MO_BSWAP, corresponding respectively to host endianness and
the opposite. Native (target) endianness in turn can be either
the host endianness, in which case MO_BSWAP is only returned for
host-opposite endianness, or the opposite, in which case 0 is only
returned for host endianness.
With this in mind, devend_memop can be compiled as a setcond+shift
for every target. Do this and, while at it, move it to
include/exec/memory.h since !NEED_CPU_H files do not (and should not)
need it.
Stefan Hajnoczi [Fri, 30 Aug 2019 09:30:56 +0000 (10:30 +0100)]
memory: fetch pmem size in get_file_size()
Neither stat(2) nor lseek(2) report the size of Linux devdax pmem
character device nodes. Commit 314aec4a6e06844937f1677f6cba21981005f389
("hostmem-file: reject invalid pmem file sizes") added code to
hostmem-file.c to fetch the size from sysfs and compare against the
user-provided size=NUM parameter:
if (backend->size > size) {
error_setg(errp, "size property %" PRIu64 " is larger than "
"pmem file \"%s\" size %" PRIu64, backend->size,
fb->mem_path, size);
return;
}
It turns out that exec.c:qemu_ram_alloc_from_fd() already has an
equivalent size check but it skips devdax pmem character devices because
lseek(2) returns 0:
if (file_size > 0 && file_size < size) {
error_setg(errp, "backing store %s size 0x%" PRIx64
" does not match 'size' option 0x" RAM_ADDR_FMT,
mem_path, file_size, size);
return NULL;
}
This patch moves the devdax pmem file size code into get_file_size() so
that we check the memory size in a single place:
qemu_ram_alloc_from_fd(). This simplifies the code and makes it more
general.
This also fixes the problem that hostmem-file only checks the devdax
pmem file size when the pmem=on parameter is given. An unchecked
size=NUM parameter can lead to SIGBUS in QEMU so we must always fetch
the file size for Linux devdax pmem character device nodes.
CONFIG_ACPI_PCI is a hard requirement of acpi-build.c, which is built
unconditionally for x86 target. Putting it in default-configs/ suggests
that it can be easily disabled, which isn't true.
Relocate the symbol with the other acpi-build.c requirements, under
'config PC'. This is similar to what is done for the arm 'virt' machine
type and CONFIG_ACPI_PCI
Paolo Bonzini [Mon, 9 Sep 2019 13:06:42 +0000 (15:06 +0200)]
test-char: fix AddressSanitizer failure
The CharSocketServerTestConfig and CharSocketClientTestConfig
objects escape after they are passed to g_test_add_data_func,
but they cease existing after the scope that defines them is
closed. Make them static to fix this issue.
Fixes: e7b6ba4186f243f149b0d8cddc129fe681ba3912 Reviewed-by: Philippe Mathieu-Daudé <[email protected]> Signed-off-by: Paolo Bonzini <[email protected]>
Wei Yang [Thu, 21 Mar 2019 08:25:53 +0000 (16:25 +0800)]
exec.c: subpage->sub_section is already initialized to 0
In subpage_init(), we will set subpage->sub_section to
PHYS_SECTION_UNASSIGNED by subpage_register. Since
PHYS_SECTION_UNASSIGNED is defined to be 0, and we allocate subpage with
g_malloc0, this means subpage->sub_section is already initialized to 0.
This patch removes the redundant setup for a new subpage and also fix
the code style.
Wanpeng Li [Mon, 15 Jul 2019 01:28:44 +0000 (09:28 +0800)]
i386/kvm: support guest access CORE cstate
Allow guest reads CORE cstate when exposing host CPU power management capabilities
to the guest. PKG cstate is restricted to avoid a guest to get the whole package
information in multi-tenant scenario.
* remotes/pmaydell/tags/pull-target-arm-20190913:
qemu-ga: Convert invocation documentation to rST
atomic_template: fix indentation in GEN_ATOMIC_HELPER
aspeed/scu: Introduce a aspeed_scu_get_apb_freq() routine
aspeed/scu: Introduce per-SoC SCU types
aspeed/smc: Calculate checksum on normal DMA
aspeed/smc: Inject errors in DMA checksum
aspeed/smc: Add DMA calibration settings
aspeed/smc: Add support for DMAs
aspeed: Use consistent typenames
aspeed: Remove unused SoC definitions
aspeed: add a GPIO controller to the SoC
hw/gpio: Add basic Aspeed GPIO model for AST2400 and AST2500
Peter Maydell [Thu, 5 Sep 2019 13:10:40 +0000 (14:10 +0100)]
qemu-ga: Convert invocation documentation to rST
The qemu-ga documentation is currently in qemu-ga.texi in
Texinfo format, which we present to the user as:
* a qemu-ga manpage
* a section of the main qemu-doc HTML documentation
Convert the documentation to rST format, and present it to
the user as:
* a qemu-ga manpage
* part of the interop/ Sphinx manual
Emulate read errors in the DMA Checksum Register for high frequencies
and optimistic settings of the Read Timing Compensation Register. This
will help in tuning the SPI timing calibration algorithm. Errors are
only injected when the property "inject_failure" is set to true as
suggested by Philippe.
The values below are those to expect from the first flash device of
the FMC controller of a palmetto-bmc machine.
When doing calibration, the SPI clock rate in the CE0 Control Register
and the read delay cycles in the Read Timing Compensation Register are
set using bit[11:4] of the DMA Control Register.
The FMC controller on the Aspeed SoCs support DMA to access the flash
modules. It can operate in a normal mode, to copy to or from the flash
module mapping window, or in a checksum calculation mode, to evaluate
the best clock settings for reads.
The model introduces two custom address spaces for DMAs: one for the
AHB window of the FMC flash devices and one for the DRAM. The latter
is populated using a "dram" link set from the machine with the RAM
container region.
Improve the naming of the different controller models to ease their
generation when initializing the SoC. The rename of the SMC types is
breaking migration compatibility.
hw/gpio: Add basic Aspeed GPIO model for AST2400 and AST2500
GPIO pins are arranged in groups of 8 pins labeled A,B,..,Y,Z,AA,AB,AC.
(Note that the ast2400 controller only goes up to group AB).
A set has four groups (except set AC which only has one) and is
referred to by the groups it is composed of (eg ABCD,EFGH,...,YZAAAB).
Each set is accessed and controlled by a bank of 14 registers.
These registers operate on a per pin level where each bit in the register
corresponds to a pin, except for the command source registers. The command
source registers operate on a per group level where bits 24, 16, 8 and 0
correspond to each group in the set.
eg. registers for set ABCD:
|D7...D0|C7...C0|B7...B0|A7...A0| <- GPIOs
|31...24|23...16|15....8|7.....0| <- bit position
Note that there are a couple of groups that only have 4 pins.
There are two ways that this model deviates from the behaviour of the
actual controller:
(1) The only control source driving the GPIO pins in the model is the ARM
model (as there currently aren't models for the LPC or Coprocessor).
(2) None of the registers in the model are reset tolerant (needs
integration with the watchdog).
Peter Maydell [Fri, 13 Sep 2019 15:04:46 +0000 (16:04 +0100)]
Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-sep-12-2019' into staging
MIPS queue for September 12th, 2019
# gpg: Signature made Thu 12 Sep 2019 17:26:10 BST
# gpg: using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <[email protected]>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01 DD75 D497 2A89 67F7 5A65
Peter Maydell [Fri, 13 Sep 2019 13:37:48 +0000 (14:37 +0100)]
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20190912a' into staging
Migration pull 2019-09-12
New feature:
UUID validation check from Yury Kotov
plus a bunch of fixes.
# gpg: Signature made Thu 12 Sep 2019 14:48:28 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <[email protected]>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20190912a:
migration: fix one typo in comment of function migration_total_bytes()
migration/qemu-file: fix potential buf waste for extra buf_index adjustment
migration/qemu-file: remove check on writev_buffer in qemu_put_compression_data
migration: Fix postcopy bw for recovery
tests/migration: Add a test for validate-uuid capability
tests/libqtest: Allow setting expected exit status
migration: Add validate-uuid capability
qemu-file: Rework old qemu_fflush comment
migration: register_savevm_live doesn't need dev
hw/net/vmxnet3: Fix leftover unregister_savevm
migration: cleanup check on ops in savevm.handlers iterations
migration: multifd_send_thread always post p->sem_sync when error happen
Peter Maydell [Fri, 13 Sep 2019 12:43:42 +0000 (13:43 +0100)]
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches:
- qcow2: Allow overwriting multiple compressed clusters at once for
better performance
- nfs: add support for nfs_umount
- file-posix: write_zeroes fixes
- qemu-io, blockdev-create, pr-manager: Fix crashes and memory leaks
- qcow2: Fix the calculation of the maximum L2 cache size
- vpc: Fix return code for vpc_co_create()
- blockjob: Code cleanup
- iotests improvements (e.g. for use with valgrind)
# gpg: Signature made Fri 13 Sep 2019 11:19:19 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <[email protected]>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream: (23 commits)
qcow2: Stop overwriting compressed clusters one by one
block/create: Do not abort if a block driver is not available
qemu-io: Don't leak pattern file in error path
iotests: extend sleeping time under Valgrind
iotests: extended timeout under Valgrind
iotests: Valgrind fails with nonexistent directory
iotests: Add casenotrun report to bash tests
iotests: exclude killed processes from running under Valgrind
iotests: allow Valgrind checking all QEMU processes
block/nfs: add support for nfs_umount
block/nfs: tear down aio before nfs_close
iotests: skip 232 when run tests as root
iotests: Test blockdev-create for vpc
iotests: Restrict nbd Python tests to nbd
iotests: Restrict file Python tests to file
iotests: Add supported protocols to execute_test()
vpc: Return 0 from vpc_co_create() on success
file-posix: Fix has_write_zeroes after NO_FALLBACK
pr-manager: Fix invalid g_free() crash bug
iotests: Test reverse sub-cluster qcow2 writes
...
Alberto Garcia [Wed, 11 Sep 2019 15:16:26 +0000 (18:16 +0300)]
qcow2: Stop overwriting compressed clusters one by one
handle_alloc() tries to find as many contiguous clusters that need
copy-on-write as possible in order to allocate all of them at the same
time.
However, compressed clusters are only overwritten one by one, so let's
say that we have an image with 1024 consecutive compressed clusters:
qemu-img create -f qcow2 hd.qcow2 64M
for f in `seq 0 64 65472`; do
qemu-io -c "write -c ${f}k 64k" hd.qcow2
done
In this case trying to overwrite the whole image with one large write
request results in 1024 separate allocations:
qemu-io -c "write 0 64M" hd.qcow2
This restriction comes from commit 095a9c58ce12afeeb90c2 from 2008.
Nowadays QEMU can overwrite multiple compressed clusters just fine,
and in fact it already does: as long as the first cluster that
handle_alloc() finds is not compressed, all other compressed clusters
in the same batch will be overwritten in one go:
qemu-img create -f qcow2 hd.qcow2 64M
qemu-io -c "write -z 0 64k" hd.qcow2
for f in `seq 64 64 65472`; do
qemu-io -c "write -c ${f}k 64k" hd.qcow2
done
Compared to the previous one, overwriting this image on my computer
goes from 8.35s down to 230ms.
block/create: Do not abort if a block driver is not available
The 'blockdev-create' QMP command was introduced as experimental
feature in commit b0292b851b8, using the assert() debug call.
It got promoted to 'stable' command in 3fb588a0f2c, but the
assert call was not removed.
Some block drivers are optional, and bdrv_find_format() might
return a NULL value, triggering the assertion.
Stable code is not expected to abort, so return an error instead.
This is easily reproducible when libnfs is not installed:
./configure
[...]
module support no
Block whitelist (rw)
Block whitelist (ro)
libiscsi support yes
libnfs support no
[...]
$ gdb qemu-system-x86_64 core
Program received signal SIGSEGV, Segmentation fault.
(gdb) bt
#0 0x00007ffff510957f in raise () at /lib64/libc.so.6
#1 0x00007ffff50f3895 in abort () at /lib64/libc.so.6
#2 0x00007ffff50f3769 in _nl_load_domain.cold.0 () at /lib64/libc.so.6
#3 0x00007ffff5101a26 in .annobin_assert.c_end () at /lib64/libc.so.6
#4 0x0000555555d7e1f1 in qmp_blockdev_create (job_id=0x555556baee40 "x", options=0x555557666610, errp=0x7fffffffc770) at block/create.c:69
#5 0x0000555555c96b52 in qmp_marshal_blockdev_create (args=0x7fffdc003830, ret=0x7fffffffc7f8, errp=0x7fffffffc7f0) at qapi/qapi-commands-block-core.c:1314
#6 0x0000555555deb0a0 in do_qmp_dispatch (cmds=0x55555645de70 <qmp_commands>, request=0x7fffdc005c70, allow_oob=false, errp=0x7fffffffc898) at qapi/qmp-dispatch.c:131
#7 0x0000555555deb2a1 in qmp_dispatch (cmds=0x55555645de70 <qmp_commands>, request=0x7fffdc005c70, allow_oob=false) at qapi/qmp-dispatch.c:174
With this patch applied, QEMU returns a QMP error:
{'execute': 'blockdev-create', 'arguments': {'job-id': 'x', 'options': {'size': 0, 'driver': 'nfs', 'location': {'path': '/', 'server': {'host': '::1', 'type': 'inet'}}}}, 'id': 'x'}
{"id": "x", "error": {"class": "GenericError", "desc": "Block driver 'nfs' not found or not supported"}}
Kevin Wolf [Tue, 10 Sep 2019 07:03:06 +0000 (09:03 +0200)]
qemu-io: Don't leak pattern file in error path
qemu_io_alloc_from_file() needs to close the pattern file even if some
error occurred.
Setting f = NULL in the success path and checking it for NULL in the
error path isn't strictly necessary at this point, but let's do it
anyway in case someone later adds a 'goto error' after closing the file.
As the iotests run longer under the Valgrind, the QEMU_COMM_TIMEOUT is
to be increased in the test cases 028, 183 and 192 when running under
the Valgrind.
iotests: Valgrind fails with nonexistent directory
The Valgrind uses the exported variable TMPDIR and fails if the
directory does not exist. Let us exclude such a test case from
being run under the Valgrind and notify the user of it.
The new function _casenotrun() is to be invoked if a test case cannot
be run for some reason. The user will be notified by a message passed
to the function. It is the caller's responsibility to make skipped a
particular test.
iotests: exclude killed processes from running under Valgrind
The Valgrind tool fails to manage its termination in multi-threaded
processes when they raise the signal SIGKILL. The bug has been reported
to the Valgrind maintainers and was registered as the bug #409141:
https://bugs.kde.org/show_bug.cgi?id=409141
Let's exclude such test cases from running under the Valgrind until a
new version with the bug fix is released because checking for the
memory issues is covered by other test cases.
iotests: allow Valgrind checking all QEMU processes
With the '-valgrind' option, let all the QEMU processes be run under
the Valgrind tool. The Valgrind own parameters may be set with its
environment variable VALGRIND_OPTS, e.g.
$ VALGRIND_OPTS="--leak-check=yes" ./check -valgrind <test#>
or they may be listed in the Valgrind checked file ./.valgrindrc or
~/.valgrindrc like
--memcheck:leak-check=no
--memcheck:track-origins=yes
To exclude a specific process from running under the Valgrind, the
corresponding environment variable VALGRIND_QEMU_<name> is to be set
to the empty string:
$ VALGRIND_QEMU_IO= ./check -valgrind <test#>
When QEMU-IO process is being killed, the shell report refers to the
text of the command in _qemu_io_wrapper(), which was modified with this
patch. So, the benchmark output for the tests 039, 061 and 137 is to be
changed also.
Multiple reports from users were received regarding failures of
packet 'g' communication with gdb for some MIPS configurations.
It was found out (by bisecting) that the problematic commit is 8e0b373. Revert that commit until a better solution is developed.
Peter Maydell [Fri, 2 Aug 2019 16:04:57 +0000 (17:04 +0100)]
target/mips: Switch to do_transaction_failed() hook
Switch the MIPS target from the old unassigned_access hook to the new
do_transaction_failed hook.
Unlike the old hook, do_transaction_failed is only ever called from
the TCG memory access paths, so there is no need for the "ignore this
if we're using KVM" hack that we were previously using to work around
the way unassigned_access was called for all kinds of memory accesses
to unassigned physical addresses.
The MIPS target does not ever do direct memory reads by physical
address (via either ldl_phys etc or address_space_ldl etc), so the
only memory accesses this affects are the 'normal' guest loads and
stores, which will be handled by the new hook; their behaviour is
unchanged.
The MIPS Jazz ('magnum' and 'pica61') boards have some code which
overrides the CPU's do_unassigned_access hook, so they can intercept
it and not raise exceptions on data accesses to invalid addresses,
only for instruction fetches.
We want to switch MIPS over to using the do_transaction_failed
hook instead, so add an intercept for that as well, and make
the board code install whichever hook the CPU is actually using.
Once we've changed the CPU implementation we can remove the
redundant code for the old hook.
Note: I am suspicious that the behaviour as implemented here may not
be what the hardware really does. It was added in commit 54e755588cf1e90f0b14 to restore the behaviour that was broken by
commit c658b94f6e8c206c59d. But prior to commit c658b94f6e8c206c59d
every MIPS board generated exceptions for instruction access to
invalid addresses but not for data accesses; and other boards,
notably Malta, were fixed by making all invalid accesses behave as
reads-as-zero (see the call to empty_slot_init() in
mips_malta_init()). Hardware that raises exceptions for instruction
access and not data access seems to me to be an unlikely design, and
it's possible that the right way to emulate this is to make the Jazz
boards do what we did with Malta (or some variation of that).
Nonetheless, since I don't have access to real hardware to test
against I have taken the approach of "make QEMU continue to behave
the same way it did before this commit". I have updated the comment
to correct the parts that are no longer accurate and note that
the hardware might behave differently.
The test case for the need for the hook-hijacking is in
https://bugs.launchpad.net/qemu/+bug/1245924 That BIOS will boot OK
either with this overriding of both hooks, or with a simple "global
memory region to ignore bad accesses of all types", so it doesn't
provide evidence either way, unfortunately.
Peter Maydell [Thu, 12 Sep 2019 15:17:26 +0000 (16:17 +0100)]
Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-4.2-pull-request' into staging
Add several floppy drive ioctl,
xtensa call0 ABI support, arm MAX_RESERVED_VA for M-profile,
aarch64 AT_HWCAP2, qOffsets' query for ELF, memfd_create,
and some code cleanup
* remotes/vivier2/tags/linux-user-for-4.2-pull-request:
linux-user: Add support for FDRESET, FDRAWCMD, FDTWADDLE, and FDEJECT ioctls
linux-user: Add support for FDMSGON and FDMSGOFF ioctls
linux-user: Add support for FDFLUSH ioctl
linux-user: Add support for FIOGETOWN and FIOSETOWN ioctls
linux-user: Add support for RNDRESEEDCRNG ioctl
linux-user: drop redundant handling of environment variables
target/xtensa: linux-user: add call0 ABI support
linux-user: Support gdb 'qOffsets' query for ELF
linux-user/arm: Adjust MAX_RESERVED_VA for M-profile
linux-user: Pass CPUState to MAX_RESERVED_VA
linux-user: add memfd_create
linux-user: fail and report on bad dfilter specs
linux-user: erroneous fd_trans_unregister call
linux-user: Add AT_HWCAP2 for aarch64-linux-user
linux-user: remove useless variable
Wei Yang [Wed, 11 Sep 2019 13:28:39 +0000 (13:28 +0000)]
migration/qemu-file: fix potential buf waste for extra buf_index adjustment
In add_to_iovec(), qemu_fflush() will be called if iovec is full. If
this happens, buf_index is reset. Currently, this is not checked and
buf_index would always been adjust with buf size.
This is not harmful, but will waste some space in file buffer.
This patch make add_to_iovec() return 1 when it has flushed the file.
Then the caller could check the return value to see whether it is
necessary to adjust the buf_index any more.
Peter Xu [Fri, 6 Sep 2019 13:01:03 +0000 (21:01 +0800)]
migration: Fix postcopy bw for recovery
We've got max-postcopy-bandwidth parameter but it's not applied
correctly after a postcopy recovery so the recovered migration stream
will still eat the whole net bandwidth. Fix that up.
Commit 78dd48df3 removed the last caller of register_savevm_live for an
instantiable device (rather than a single system wide device);
so trim out the parameter.
Commit 78dd48df3 reworked vmxnet3's live migration but left a straggling
unregister_savevm call. Remove it, although it doesn't seem to have
any bad effect.
Wei Yang [Mon, 19 Aug 2019 03:28:04 +0000 (11:28 +0800)]
migration: cleanup check on ops in savevm.handlers iterations
During migration, there are several places to iterate on
savevm.handlers. And on each iteration, we need to check its ops and
related callbacks before invoke it.
Generally, ops is the first element to check, and it is only necessary
to check it once.
This patch clean all the related part in savevm.c to check ops only once
in those iterations.
Ivan Ren [Thu, 29 Aug 2019 02:16:36 +0000 (10:16 +0800)]
migration: multifd_send_thread always post p->sem_sync when error happen
When encounter error, multifd_send_thread should always notify who pay
attention to it before exit. Otherwise it may block migration_thread
at multifd_send_sync_main forever.
Error as follow:
-------------------------------------------------------------------------------
(gdb) bt
#0 0x00007f4d669dfa0b in do_futex_wait.constprop.1 () from /lib64/libpthread.so.0
#1 0x00007f4d669dfa9f in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0
#2 0x00007f4d669dfb3b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0
#3 0x0000562ccf0a5614 in qemu_sem_wait (sem=sem@entry=0x562cd1b698e8) at util/qemu-thread-posix.c:319
#4 0x0000562ccecb4752 in multifd_send_sync_main (rs=<optimized out>) at /qemu/migration/ram.c:1099
#5 0x0000562ccecb95f4 in ram_save_iterate (f=0x562cd0ecc000, opaque=<optimized out>) at /qemu/migration/ram.c:3550
#6 0x0000562ccef43c23 in qemu_savevm_state_iterate (f=0x562cd0ecc000, postcopy=false) at migration/savevm.c:1189
#7 0x0000562ccef3dcf3 in migration_iteration_run (s=0x562cd09fabf0) at migration/migration.c:3131
#8 migration_thread (opaque=opaque@entry=0x562cd09fabf0) at migration/migration.c:3258
#9 0x0000562ccf0a4c26 in qemu_thread_start (args=<optimized out>) at util/qemu-thread-posix.c:502
#10 0x00007f4d669d9e25 in start_thread () from /lib64/libpthread.so.0
#11 0x00007f4d6670635d in clone () from /lib64/libc.so.6
(gdb) f 4
#4 0x0000562ccecb4752 in multifd_send_sync_main (rs=<optimized out>) at /qemu/migration/ram.c:1099
1099 qemu_sem_wait(&p->sem_sync);
(gdb) list
1094 }
1095 for (i = 0; i < migrate_multifd_channels(); i++) {
1096 MultiFDSendParams *p = &multifd_send_state->params[i];
1097
1098 trace_multifd_send_sync_main_wait(p->id);
1099 qemu_sem_wait(&p->sem_sync);
1100 }
1101 trace_multifd_send_sync_main(multifd_send_state->packet_num);
1102 }
1103
(gdb) p i
$1 = 0
(gdb) p multifd_send_state->params[0].pending_job
$2 = 2 //It means the job before MULTIFD_FLAG_SYNC has already fail
(gdb) p multifd_send_state->params[0].quit
$3 = true
tests: make filemonitor test more robust to event ordering
The ordering of events that are emitted during the rmdir
test have changed with kernel >= 5.3. Semantically both
new & old orderings are correct, so we must be able to
cope with either.
To cope with this, when we see an unexpected event, we
push it back onto the queue and look and the subsequent
event to see if that matches instead.
Yunqiang Su [Wed, 4 Sep 2019 12:59:24 +0000 (14:59 +0200)]
linux-user: Add support for FDFLUSH ioctl
FDFLUSH is used for flushing buffers of floppy drives. Support in
QEMU is needed because some of Debian packages use this ioctl while
running post-build tests. One such example is 'tar' package.
projects / qemu.git / log