Michael Roth [Thu, 12 Sep 2019 23:12:01 +0000 (18:12 -0500)]
make-release: pull in edk2 submodules so we can build it from tarballs
The `make efi` target added by 536d2173 is built from the roms/edk2
submodule, which in turn relies on additional submodules nested under
roms/edk2.
The make-release script currently only pulls in top-level submodules,
so these nested submodules are missing in the resulting tarball.
We could try to address this situation more generally by recursively
pulling in all submodules, but this doesn't necessarily ensure the
end-result will build properly (this case also required other changes).
Additionally, due to the nature of submodules, we may not always have
control over how these sorts of things are dealt with, so for now we
continue to handle it on a case-by-case in the make-release script.
* remotes/bonzini/tags/for-upstream: (29 commits)
target/i386/kvm: Silence warning from Valgrind about uninitialized bytes
target/i386: work around KVM_GET_MSRS bug for secondary execution controls
target/i386: add VMX features
vmxcap: correct the name of the variables
target/i386: add VMX definitions
target/i386: expand feature words to 64 bits
target/i386: introduce generic feature dependency mechanism
target/i386: handle filtered_features in a new function mark_unavailable_features
tests/docker: only enable ubsan for test-clang
win32: work around main-loop busy loop on socket/fd event
tests: skip serial test on windows
util: WSAEWOULDBLOCK on connect should map to EINPROGRESS
Fix wrong behavior of cpu_memory_rw_debug() function in SMM
memory: allow memory_region_register_iommu_notifier() to fail
vfio: Turn the container error into an Error handle
i386: Add CPUID bit for CLZERO and XSAVEERPTR
docker: test-debug: disable LeakSanitizer
lm32: do not leak memory on object_new/object_unref
cris: do not leak struct cris_disasm_data
mips: fix memory leaks in board initialization
...
Thomas Huth [Tue, 24 Sep 2019 07:47:38 +0000 (09:47 +0200)]
target/i386/kvm: Silence warning from Valgrind about uninitialized bytes
When I run QEMU with KVM under Valgrind, I currently get this warning:
Syscall param ioctl(generic) points to uninitialised byte(s)
at 0x95BA45B: ioctl (in /usr/lib64/libc-2.28.so)
by 0x429DC3: kvm_ioctl (kvm-all.c:2365)
by 0x51B249: kvm_arch_get_supported_msr_feature (kvm.c:469)
by 0x4C2A49: x86_cpu_get_supported_feature_word (cpu.c:3765)
by 0x4C4116: x86_cpu_expand_features (cpu.c:5065)
by 0x4C7F8D: x86_cpu_realizefn (cpu.c:5242)
by 0x5961F3: device_set_realized (qdev.c:835)
by 0x7038F6: property_set_bool (object.c:2080)
by 0x707EFE: object_property_set_qobject (qom-qobject.c:26)
by 0x705814: object_property_set_bool (object.c:1338)
by 0x498435: pc_new_cpu (pc.c:1549)
by 0x49C67D: pc_cpus_init (pc.c:1681)
Address 0x1ffeffee74 is on thread 1's stack
in frame #2, created by kvm_arch_get_supported_msr_feature (kvm.c:445)
It's harmless, but a little bit annoying, so silence it by properly
initializing the whole structure with zeroes.
Paolo Bonzini [Tue, 2 Jul 2019 12:58:48 +0000 (14:58 +0200)]
target/i386: work around KVM_GET_MSRS bug for secondary execution controls
Some secondary controls are automatically enabled/disabled based on the CPUID
values that are set for the guest. However, they are still available at a
global level and therefore should be present when KVM_GET_MSRS is sent to
/dev/kvm.
Unfortunately KVM forgot to include those, so fix that.
Paolo Bonzini [Mon, 1 Jul 2019 16:32:17 +0000 (18:32 +0200)]
target/i386: add VMX features
Add code to convert the VMX feature words back into MSR values,
allowing the user to enable/disable VMX features as they wish. The same
infrastructure enables support for limiting VMX features in named
CPU models.
Paolo Bonzini [Mon, 1 Jul 2019 15:38:54 +0000 (17:38 +0200)]
target/i386: expand feature words to 64 bits
VMX requires 64-bit feature words for the IA32_VMX_EPT_VPID_CAP
and IA32_VMX_BASIC MSRs. (The VMX control MSRs are 64-bit wide but
actually have only 32 bits of information).
Sometimes a CPU feature does not make sense unless another is
present. In the case of VMX features, KVM does not even allow
setting the VMX controls to some invalid combinations.
Therefore, this patch adds a generic mechanism that looks for bits
that the user explicitly cleared, and uses them to remove other bits
from the expanded CPU definition. If these dependent bits were also
explicitly *set* by the user, this will be a warning for "-cpu check"
and an error for "-cpu enforce". If not, then the dependent bits are
cleared silently, for convenience.
With VMX features, this will be used so that for example
"-cpu host,-rdrand" will also hide support for RDRAND exiting.
Paolo Bonzini [Tue, 2 Jul 2019 13:32:41 +0000 (15:32 +0200)]
target/i386: handle filtered_features in a new function mark_unavailable_features
The next patch will add a different reason for filtering features, unrelated
to host feature support. Extract a new function that takes care of disabling
the features and optionally reporting them.
Paolo Bonzini [Tue, 1 Oct 2019 13:48:55 +0000 (15:48 +0200)]
tests/docker: only enable ubsan for test-clang
-fsanitize=undefined is not the same thing as --enable-sanitizers. After
commit 47c823e ("tests/docker: add sanitizers back to clang build", 2019-09-11)
test-clang is almost duplicating the asan (test-debug) test, so
partly revert commit 47c823e5b while leaving ubsan enabled.
The ctx->notifier event is added to the gpoll sources in
aio_set_event_notifier(), aio_ctx_check() should clear the event
regardless of ctx->notified, since Windows sets the event by itself,
bypassing the aio->notified. This fixes qemu not clearing the event
resulting in a busy loop.
Paolo suggested to me on irc to call event_notifier_test_and_clear()
after select() >0 from aio-win32.c's aio_prepare. Unfortunately, not all
fds associated with ctx->notifiers are in AIO fd handlers set.
(qemu_set_nonblock() in util/oslib-win32.c calls qemu_fd_register()).
This is essentially a v2 of a patch that was sent earlier:
https://lists.gnu.org/archive/html/qemu-devel/2017-01/msg00420.html
that resurfaced when James investigated Spice performance issues on Windows:
https://gitlab.freedesktop.org/spice/spice/issues/36
In order to test that patch, I simply tried running test-char on
win32, and it hangs. Applying that patch solves it. QIO idle sources
are not dispatched. I haven't investigated much further, I suspect
source priorities and busy looping still come into play.
This version keeps the "notified" field, so event_notifier_poll()
should still work as expected.
util: WSAEWOULDBLOCK on connect should map to EINPROGRESS
In general, WSAEWOULDBLOCK can be mapped to EAGAIN as done by
socket_error() (or EWOULDBLOCK). But for connect() with non-blocking
sockets, it actually means the operation is in progress:
https://docs.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-connect
"The socket is marked as nonblocking and the connection cannot be completed immediately."
(this is also the behaviour implemented by GLib GSocket)
This fixes socket_can_bind_connect() test on win32.
Fix wrong behavior of cpu_memory_rw_debug() function in SMM
There is a problem, that you don't have access to the data using cpu_memory_rw_debug() function when in SMM. You can't remotely debug SMM mode program because of that for example.
Likely attrs version of get_phys_page_debug should be used to get correct asidx at the end to handle access properly.
Here the patch to fix it.
Eric Auger [Tue, 24 Sep 2019 08:25:17 +0000 (10:25 +0200)]
memory: allow memory_region_register_iommu_notifier() to fail
Currently, when a notifier is attempted to be registered and its
flags are not supported (especially the MAP one) by the IOMMU MR,
we generally abruptly exit in the IOMMU code. The failure could be
handled more nicely in the caller and especially in the VFIO code.
So let's allow memory_region_register_iommu_notifier() to fail as
well as notify_flag_changed() callback.
All sites implementing the callback are updated. This patch does
not yet remove the exit(1) in the amd_iommu code.
in SMMUv3 we turn the warning message into an error message saying
that the assigned device would not work properly.
Eric Auger [Tue, 24 Sep 2019 08:25:16 +0000 (10:25 +0200)]
vfio: Turn the container error into an Error handle
The container error integer field is currently used to store
the first error potentially encountered during any
vfio_listener_region_add() call. However this fails to propagate
detailed error messages up to the vfio_connect_container caller.
Instead of using an integer, let's use an Error handle.
Messages are slightly reworded to accomodate the propagation.
Paolo Bonzini [Tue, 1 Oct 2019 13:36:28 +0000 (15:36 +0200)]
docker: test-debug: disable LeakSanitizer
There are just too many leaks in device-introspect-test (especially for
the plethora of arm and aarch64 boards) to make LeakSanitizer useful;
disable it for now.
Whoever is interested in debugging leaks can also use valgrind like this:
Paolo Bonzini [Tue, 1 Oct 2019 13:36:27 +0000 (15:36 +0200)]
lm32: do not leak memory on object_new/object_unref
Bottom halves and ptimers are malloced, but nothing in these
files is freeing memory allocated by instance_init. Since
these are sysctl devices that are never unrealized, just moving
the allocations to realize is enough to avoid the leak in
practice (and also to avoid upsetting asan when running
device-introspect-test).
Thomas Huth [Mon, 30 Sep 2019 15:04:36 +0000 (17:04 +0200)]
hw/isa: Introduce a CONFIG_ISA_SUPERIO switch for isa-superio.c
Currently, isa-superio.c is always compiled as soon as CONFIG_ISA_BUS
is enabled. But there are also machines that have an ISA BUS without
any of the superio chips attached to it, so we should not compile
isa-superio.c in case we only compile a QEMU for such a machine.
Thus add a proper CONFIG_ISA_SUPERIO switch so that this file only gets
compiled when we really, really need it.
Pavel Dovgalyuk [Tue, 17 Sep 2019 09:54:06 +0000 (12:54 +0300)]
replay: don't synchronize memory operations in replay mode
Commit 9458a9a1df1a4c719e24512394d548c1fc7abd22 added synchronization
of vCPU and migration operations through calling run_on_cpu operation.
However, in replay mode this synchronization is unneeded, because
I/O and vCPU threads are already synchronized.
This patch disables such synchronization for record/replay mode.
qemu-pr-helper: fix crash in mpath_reconstruct_sense
The 'r' variable was accidently shadowed, and because of this
we were always passing 0 to mpath_generic_sense, instead of original
return value, which triggers an abort()
This is an attempt to fix the
https://bugzilla.redhat.com/show_bug.cgi?id=1720047
although there might be other places in the code
that trigger qemu-pr-helper crash, and this fix might
not be the root cause.
The crash was reproduced by creating an iscsi target on a test machine,
and passing it twice to the guest like that:
Strictly speaking this is wrong configuration since qemu is where
the multipath was split, and thus the iscsi target was not aware of
multipath, and thus when libmpathpersist code rightfully tried to register
the PR key on all paths, it failed to do so.
However qemu-pr-helper should not crash in this case.
Alex Bennée [Wed, 2 Oct 2019 10:22:12 +0000 (11:22 +0100)]
accel/kvm: ensure ret always set
Some of the cross compilers rightly complain there are cases where ret
may not be set. 0 seems to be the reasonable default unless particular
slot explicitly returns -1.
Peter Maydell [Tue, 1 Oct 2019 15:21:42 +0000 (16:21 +0100)]
Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-oct-01-2019' into staging
MIPS queue for October 1st, 2019
# gpg: Signature made Tue 01 Oct 2019 16:10:43 BST
# gpg: using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <[email protected]>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01 DD75 D497 2A89 67F7 5A65
* remotes/amarkovic/tags/mips-queue-oct-01-2019:
target/mips: msa: Move helpers for <AND|NOR|OR|XOR>.V
target/mips: msa: Simplify and move helper for MOVE.V
target/mips: msa: Split helpers for MOD_<S|U>.<B|H|W|D>
target/mips: msa: Split helpers for DIV_<S|U>.<B|H|W|D>
target/mips: msa: Split helpers for CLT_<S|U>.<B|H|W|D>
target/mips: msa: Split helpers for CLE_<S|U>.<B|H|W|D>
target/mips: msa: Split helpers for CEQ.<B|H|W|D>
target/mips: msa: Split helpers for AVER_<S|U>.<B|H|W|D>
target/mips: msa: Split helpers for AVE_<S|U>.<B|H|W|D>
target/mips: msa: Split helpers for B<CLR|NEG|SEL>.<B|H|W|D>
target/mips: msa: Unroll loops and demacro <BMNZ|BMZ|BSEL>.V
target/mips: msa: Split helpers for BINS<L|R>.<B|H|W|D>
target/mips: msa: Split helpers for PCNT.<B|H|W|D>
target/mips: msa: Split helpers for <NLOC|NLZC>.<B|H|W|D>
target/mips: Clean up translate.c
target/mips: Clean up mips-defs.h
target/mips: Clean up kvm_mips.h
target/mips: Clean up internal.h
Peter Maydell [Tue, 1 Oct 2019 12:13:38 +0000 (13:13 +0100)]
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-10-01' into staging
- Fix and re-enable the usb-hcd-ehci-test
- Silence a Coverity warning in hw/m68k/next-cube.c
- Fix crash that can occur when using bad binaries with "-kernel"
- Disallow colons in the "-accel" parameter
* remotes/huth-gitlab/tags/pull-request-2019-10-01:
Disallow colons in the parameter of "-accel"
hw/core/loader: Fix possible crash in rom_copy()
hw/m68k/next-cube: Avoid static RTC variables and introduce control register
tests: fix echi/ehci typo
tests: fix usb-hcd-ehci-test compilation
* remotes/armbru/tags/pull-qapi-2019-09-28: (27 commits)
qapi: Improve source file read error handling
qapi: Improve reporting of redefinition
qapi: Improve reporting of missing documentation comment
qapi: Eliminate check_keys(), rename check_known_keys()
qapi: Improve reporting of invalid 'if' further
qapi: Avoid redundant definition references in error messages
qapi: Improve reporting of missing / unknown definition keys
qapi: Improve reporting of invalid flags
qapi: Improve reporting of invalid 'if' errors
qapi: Move context-free checking to the proper place
qapi: Move context-sensitive checking to the proper place
qapi: Inline check_name() into check_union()
qapi: Plumb info to the QAPISchemaMember
qapi: Make check_type()'s array case a bit more obvious
qapi: Move check for reserved names out of add_name()
qapi: Report invalid '*' prefix like any other invalid name
qapi: Use check_name_str() where it suffices
qapi: Improve reporting of invalid name errors
qapi: Reorder check_FOO() parameters for consistency
qapi: Improve reporting of member name clashes
...
Thomas Huth [Mon, 23 Sep 2019 12:00:29 +0000 (14:00 +0200)]
Disallow colons in the parameter of "-accel"
Everybody who used something like "-machine accel=kvm:tcg" in the past
might be tempted to specify a similar list with the -accel parameter,
too, for example "-accel kvm:tcg". However, this is not how this
options is thought to be used, since each "-accel" should only take care
of one specific accelerator.
In the long run, we really should rework the "-accel" code completely,
so that it does not set "-machine accel=..." anymore internally, but
is completely independent from "-machine". For the short run, let's
make sure that users cannot use "-accel xyz:tcg", so that we avoid
that we have to deal with such cases in the wild later.
Thomas Huth [Wed, 25 Sep 2019 12:16:43 +0000 (14:16 +0200)]
hw/core/loader: Fix possible crash in rom_copy()
Both, "rom->addr" and "addr" are derived from the binary image
that can be loaded with the "-kernel" paramer. The code in
rom_copy() then calculates:
d = dest + (rom->addr - addr);
and uses "d" as destination in a memcpy() some lines later. Now with
bad kernel images, it is possible that rom->addr is smaller than addr,
thus "rom->addr - addr" gets negative and the memcpy() then tries to
copy contents from the image to a bad memory location. This could
maybe be used to inject code from a kernel image into the QEMU binary,
so we better fix it with an additional sanity check here.
Thomas Huth [Sat, 21 Sep 2019 09:17:38 +0000 (11:17 +0200)]
hw/m68k/next-cube: Avoid static RTC variables and introduce control register
Coverity currently complains that the "if (0x00 & (0x80 >> (phase - 8))"
in next-cube.c can never be true. Right it is. The "0x00" is meant as value
of the control register of the RTC, which is currently not implemented yet.
Thus, let's add a register variable for this now. However, the RTC
registers are currently defined as static variables in nextscr2_write(),
which is quite ugly. Thus let's also move the RTC variables to the main
machine state instead. In the long run, we should likely even refactor
the whole RTC code into a separate device in a separate file, but that's
something for calm winter nights later... as a first step, cleaning up
the static variables and shutting up the warning from Coverity should
be sufficient.
Peter Maydell [Mon, 30 Sep 2019 13:21:56 +0000 (14:21 +0100)]
Merge remote-tracking branch 'remotes/borntraeger/tags/s390x-20190930' into staging
- do not abuse memory_region_allocate_system_memory and split the memory
according to KVM memslots in KVM code instead (Paolo, Igor)
- change splitting to split at 4TB (Christian)
- do not claim s390 (31bit) support in configure (Thomas)
- sclp error checking (Janosch, Claudio)
- new s390 pci maintainer (Matt, Collin)
- fix s390 pci (again) (Matt)
* remotes/borntraeger/tags/s390x-20190930:
s390/kvm: split kvm mem slots at 4TB
s390: do not call memory_region_allocate_system_memory() multiple times
kvm: split too big memory section on several memslots
kvm: clear dirty bitmaps from all overlapping memslots
kvm: extract kvm_log_clear_one_slot
configure: Remove s390 (31-bit mode) from the list of supported CPUs
s390x: sclp: Report insufficient SCCB length
s390x: sclp: fix error handling for oversize control blocks
s390x: sclp: boundary check
s390x: sclp: refactor invalid command check
s390: PCI: fix IOMMU region init
MAINTAINERS: Update S390 PCI Maintainer
Igor Mammedov [Tue, 24 Sep 2019 14:47:51 +0000 (10:47 -0400)]
s390: do not call memory_region_allocate_system_memory() multiple times
s390 was trying to solve limited KVM memslot size issue by abusing
memory_region_allocate_system_memory(), which breaks API contract
where the function might be called only once.
Beside an invalid use of API, the approach also introduced migration
issue, since RAM chunks for each KVM_SLOT_MAX_BYTES are transferred in
migration stream as separate RAMBlocks.
After discussion [1], it was agreed to break migration from older
QEMU for guest with RAM >8Tb (as it was relatively new (since 2.12)
and considered to be not actually used downstream).
Migration should keep working for guests with less than 8TB and for
more than 8TB with QEMU 4.2 and newer binary.
In case user tries to migrate more than 8TB guest, between incompatible
QEMU versions, migration should fail gracefully due to non-exiting
RAMBlock ID or RAMBlock size mismatch.
Taking in account above and that now KVM code is able to split too
big MemorySection into several memslots, partially revert commit
(bb223055b s390-ccw-virtio: allow for systems larger that 7.999TB)
and use kvm_set_max_memslot_size() to set KVMSlot size to
KVM_SLOT_MAX_BYTES.
1) [PATCH RFC v2 4/4] s390: do not call memory_region_allocate_system_memory() multiple times
Igor Mammedov [Tue, 24 Sep 2019 14:47:50 +0000 (10:47 -0400)]
kvm: split too big memory section on several memslots
Max memslot size supported by kvm on s390 is 8Tb,
move logic of splitting RAM in chunks upto 8T to KVM code.
This way it will hide KVM specific restrictions in KVM code
and won't affect board level design decisions. Which would allow
us to avoid misusing memory_region_allocate_system_memory() API
and eventually use a single hostmem backend for guest RAM.
Paolo Bonzini [Tue, 24 Sep 2019 14:47:49 +0000 (10:47 -0400)]
kvm: clear dirty bitmaps from all overlapping memslots
Currently MemoryRegionSection has 1:1 mapping to KVMSlot.
However next patch will allow splitting MemoryRegionSection into
several KVMSlot-s, make sure that kvm_physical_log_slot_clear()
is able to handle such 1:N mapping.
Thomas Huth [Sat, 28 Sep 2019 19:03:34 +0000 (21:03 +0200)]
configure: Remove s390 (31-bit mode) from the list of supported CPUs
On IBM Z, KVM in the kernel is only implemented for 64-bit mode, and
with regards to TCG, we also only support 64-bit host CPUs (see the
check at the beginning of tcg/s390/tcg-target.inc.c), so we should
remove s390 (without "x", i.e. the old 31-bit mode CPUs) from the
list of supported CPUs.
Return the correct error code when the SCCB buffer is too small to
contain all of the output, for the Read SCP Information and
Read CPU Information commands.
Matthew Rosato [Thu, 26 Sep 2019 14:10:36 +0000 (10:10 -0400)]
s390: PCI: fix IOMMU region init
The fix in dbe9cf606c shrinks the IOMMU memory region to a size
that seems reasonable on the surface, however is actually too
small as it is based against a 0-mapped address space. This
causes breakage with small guests as they can overrun the IOMMU window.
Let's go back to the prior method of initializing iommu for now.
Peter Maydell [Mon, 30 Sep 2019 10:02:22 +0000 (11:02 +0100)]
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190927' into staging
target-arm queue:
* Fix the CBAR register implementation for Cortex-A53,
Cortex-A57, Cortex-A72
* Fix direct booting of Linux kernels on emulated CPUs
which have an AArch32 EL3 (incorrect NSACR settings
meant they could not access the FPU)
* semihosting cleanup: do more work at translate time
and less work at runtime
* remotes/pmaydell/tags/pull-target-arm-20190927:
hw/arm/boot: Use the IEC binary prefix definitions
hw/arm/boot.c: Set NSACR.{CP11,CP10} for NS kernel boots
tests/tcg: add linux-user semihosting smoke test for ARM
target/arm: remove run-time semihosting checks for linux-user
target/arm: remove run time semihosting checks
target/arm: handle A-profile semihosting at translate time
target/arm: handle M-profile semihosting at translate time
tests/tcg: clean-up some comments after the de-tangling
target/arm: fix CBAR register for AArch64 CPUs
Signed-off-by: Peter Maydell <[email protected]>
# Conflicts:
# tests/tcg/arm/Makefile.target
qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file. Lazy.
Change QAPISchema.__init__() to take a file name instead of a file
object. Move the open code from _include() to __init__(), so it's
used for the main schema file, too.
Move the read into the try for good measure, and rephrase the error
message.
Reporting open or read failure for the main schema file needs a
QAPISourceInfo representing "no source". Make QAPISourceInfo cope
with fname=None.
qapi: Improve reporting of missing / unknown definition keys
Have check_exprs() call check_keys() later, so its error messages gain
an "in definition" line.
Both check_keys() and check_name_is_str() check the definition's name
is a string. Since check_keys() now runs after check_name_is_str()
rather than before, its check is dead. Bury it. Checking values in
check_keys() is unclean anyway.
Split check_flags() off check_keys() and have check_exprs() call it
later, so its error messages gain an "in definition" line. Tweak the
error messages.
Checking values in a function named check_keys() is unclean anyway.
Move check_if() from check_keys() to check_exprs() and call it later,
so its error messages gain an "in definition" line.
Checking values in a function named check_keys() is unclean anyway.
The original sin was commit 0545f6b887 "qapi: Better error messages
for bad expressions", which checks the value of key 'name'. More
sinning in commit 2cbf09925a "qapi: More rigorous checking for type
safety bypass", commit c818408e44 "qapi: Implement boxed types for
commands/events", and commit 967c885108 "qapi: add 'if' to top-level
expressions". This commit does penance for the latter. The next
commits will do penance for the others.
qapi: Move context-free checking to the proper place
QAPISchemaCommand.check() and QAPISchemaEvent().check() check 'data'
is present when 'boxed': true. That's context-free. Move to
check_command() and check_event().
Tweak the error message while there.
check_exprs() & friends now check exactly what qapi-code-gen.txt calls
the second layer of syntax.
qapi: Move context-sensitive checking to the proper place
When we introduced the QAPISchema intermediate representation (commit ac88219a6c7), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check(). The .check() assert check_exprs() did its job.
Time to finish the conversion job. Move exactly the context-sensitive
checks to the .check(). They replace assertions there. Context-free
checks stay put.
Fixes the misleading optional tag error demonstrated by test
flat-union-optional-discriminator.
check_name() consists of check_name_is_str() and check_name_str().
check_union() relies on the latter to catch optional discriminators.
The next commit will replace that by a more straightforward check.
Inlining check_name() into check_union() now should make that easier
to review.
qapi: Make check_type()'s array case a bit more obvious
check_type() checks the array's contents, then peels off the array and
falls through to the "not array" code without resetting allow_array
and allow_dict to False. Works because the peeled value is a string,
and allow_array and allow_dict aren't used then. Tidy up anyway:
recurse instead, defaulting allow_array and allow_dict to False.
qapi: Move check for reserved names out of add_name()
The checks for reserved names are spread far and wide. Move one from
add_name() to new check_defn_name_str(). This is a first step towards
collecting them all in dedicated name checking functions next to
check_name().
While there, drop the quotes around the meta-type in
check_name_str()'s error messages: "'command' uses ... name 'NAME'"
becomes "command uses ... name 'NAME'".
struct-base-clash.json: In struct 'Sub':
struct-base-clash.json:5: 'name' (member of Sub) collides with 'name' (member of Base)
The "(member of Sub)" is redundant with "In struct 'Sub'". Comes from
QAPISchemaMember.describe(). Pass info to it, so it can detect the
redundancy and avoid it. Result:
struct-base-clash.json: In struct 'Sub':
struct-base-clash.json:5: member 'name' collides with member 'name' of type 'Base'
qapi: Change frontend error messages to start with lower case
Starting error messages with a capital letter complicates things when
text can get interpolated both at the beginning and in the middle of
an error message. The next patch will do that. Switch to lower case
to keep it simpler.
For what it's worth, the GNU Coding Standards advise the message
"should not begin with a capital letter when it follows a program name
and/or file name, because that isn’t the beginning of a sentence. (The
sentence conceptually starts at the beginning of the line.)"
While there, avoid breaking lines containing multiple arguments in the
middle of an argument.
QAPISchemaMember.check_clash() checks for member names that map to the
same c_name(). Takes care of rejecting duplicate names.
It also checks a naming rule: no uppercase in member names. That's a
rather odd place to do it. Enforcing naming rules is
check_name_str()'s job.
qapi-code-gen.txt specifies the name case rule applies to the name as
it appears in the schema. check_clash() checks c_name(name) instead.
No difference, as c_name() leaves alone case, but unclean.
Move the name case check into check_name_str(), less the c_name().
New argument @permit_upper suppresses it. Pass permit_upper=True for
definitions (which are not members), and when the member's owner is
whitelisted with pragma name-case-whitelist.
Bonus: name-case-whitelist now applies to a union's inline base, too.
Update qapi/qapi-schema.json pragma to whitelist union CpuInfo instead
of CpuInfo's implicit base type's name q_obj_CpuInfo-base.
qapi: Prefix frontend errors with an "in definition" line
We take pains to include the offending expression in error messages,
e.g.
tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'
But not always:
tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings
Instead of improving them one by one, report the offending expression
whenever it is known, like this:
tests/qapi-schema/enum-if-invalid.json: In enum 'TestIfEnum':
tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings
Error messages that mention the offending expression become a bit
redundant, e.g.
tests/qapi-schema/alternate-any.json: In alternate 'Alt':
tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'
QAPISchemaMember.owner is the name of the defining entity. That's a
confusing name when an object type inherits members from a base type.
Rename it to .defined_in. Rename .set_owner() and ._pretty_owner() to
match.
When we introduced the QAPISchema intermediate representation (commit ac88219a6c7), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check(). check_exprs() still checks and reports errors,
and the .check() assert check_exprs() did the job. There are a few
gaps, though.
QAPISchemaArrayType.check() neglects to assert the element type is not
an array. Add the assertion.
QAPISchemaObjectTypeVariants.check() neglects to assert the tag member
is not optional. Add the assertion.
It neglects to assert the tag member is not conditional. Add the
assertion.
It neglects to assert we actually have variants. Add the assertion.
It asserts the variants are object types, but neglects to assert they
don't have variants. Tighten the assertion.
QAPISchemaObjectTypeVariants.check_clash() has the same issue.
However, it can run only after .check(). Delete the assertion instead
of tightening it.
QAPISchemaAlternateType.check() neglects to assert the branch types
don't conflict. Fixing that isn't trivial, so add just a TODO comment
for now. It'll be resolved later in this series.
Michal Privoznik [Fri, 30 Aug 2019 13:29:45 +0000 (15:29 +0200)]
qmp-dispatch: Use CommandNotFound error for disabled commands
If a command is disabled an error is reported. But due to usage of
error_setg() the class of the error is GenericError which does not
help callers in distinguishing this case from a case where a qmp
command fails regularly due to other reasons.
We used to use class CommandDisabled until the great error
simplification (commit de253f1491 for QMP and commit 93b91c59db for
qemu-ga, both v1.2.0).
Use CommandNotFound error class, which is close enough.
Peter Maydell [Fri, 27 Sep 2019 15:53:29 +0000 (16:53 +0100)]
Merge remote-tracking branch 'remotes/berrange/tags/qcrypto-next-pull-request' into staging
Refactoring of LUKS support to facilitate keyslot updates
No current functional change is expected with this series.
# gpg: Signature made Fri 27 Sep 2019 10:58:54 BST
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <[email protected]>" [full]
# gpg: aka "Daniel P. Berrange <[email protected]>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF
* remotes/berrange/tags/qcrypto-next-pull-request:
qcrypto-luks: more rigorous header checking
qcrypto-luks: simplify the math used for keyslot locations
qcrypto-luks: extract store key function
qcrypto-luks: extract check and parse header
qcrypto-luks: extract store and load header
qcrypto-luks: purge unused error codes from open callback
qcrypto-luks: use the parsed encryption settings in QCryptoBlockLUKS
qcrypto-luks: pass keyslot index rather that pointer to the keyslot
qcrypto-luks: simplify masterkey and masterkey length
qcrypto-luks: don't overwrite cipher_mode in header
qcrypto-luks: rename some fields in QCryptoBlockLUKSHeader
projects / qemu.git / log