Peter Maydell [Wed, 20 Jul 2016 17:52:10 +0000 (18:52 +0100)]
Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging
# gpg: Signature made Wed 20 Jul 2016 12:19:56 BST
# gpg: using RSA key 0xCA35624C6A9171C6
# gpg: Good signature from "Fam Zheng <[email protected]>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5003 7CB7 9706 0F76 F021 AD56 CA35 624C 6A91 71C6
* remotes/famz/tags/docker-pull-request:
docker: pass EXECUTABLE to build script
docker: Don't start a container that doesn't exist
docker: Add "images" subcommand to docker.py
docker: Fix exit code if $CMD failed
docker: More sensible run script
tests/docker/docker.py: add update operation
tests/docker/dockerfiles: new debian-bootstrap.docker
tests/docker/docker.py: check and run .pre script
tests/docker/docker.py: support --include-executable
tests/docker/docker.py: docker_dir outside build
* remotes/cody/tags/block-pull-request:
block/gluster: add support for multiple gluster servers
block/gluster: using new qapi schema
block/gluster: deprecate rdma support
block/gluster: code cleanup
block/gluster: rename [server, volname, image] -> [host, volume, path]
mirror: fix request throttling in drive-mirror
mirror: improve performance of mirroring of empty disk
mirror: efficiently zero out target
mirror: optimize dirty bitmap filling in mirror_run a bit
block: remove extra condition in bdrv_can_write_zeroes_with_unmap
mirror: create mirror_dirty_init helper for mirror_run
mirror: create mirror_throttle helper
mirror: make sectors_in_flight int64_t
dirty-bitmap: operate with int64_t amount
* remotes/armbru/tags/pull-qapi-2016-07-19:
net: Use correct type for bool flag
qapi: Change Netdev into a flat union
block: Simplify drive-mirror
block: Simplify block_set_io_throttle
qapi: Implement boxed types for commands/events
qapi: Plumb in 'boxed' to qapi generator lower levels
qapi-event: Simplify visit of non-implicit data
qapi: Drop useless gen_err_check()
qapi: Add type.is_empty() helper
qapi: Hide tag_name data member of variants
qapi: Special case c_name() for empty type
qapi: Require all branches of flat union enum to be covered
net: use Netdev instead of NetClientOptions in client init
qapi: change QmpInputVisitor to QSLIST
qapi: change QmpOutputVisitor to QSLIST
* remotes/pmaydell/tags/pull-target-arm-20160719:
arm_gicv3: Add assert()s to tell Coverity that offsets are aligned
target-arm: Fix unreachable code in gicv3_class_name()
Alex Bennée [Tue, 19 Jul 2016 13:20:45 +0000 (21:20 +0800)]
docker: pass EXECUTABLE to build script
To build a docker image with which needs qemu linux-user emulation we
need to pass --include-executable to the build script. Using the same
mechanism as for other container controls we enable the option is
EXECUTABLE is set on the make command line e.g:
make docker-image-debian-bootstrap V=1 J=9 DEB_ARCH=armhf \
DEB_TYPE=stable EXECUTABLE=./arm-linux-user/qemu-arm
docker: Don't start a container that doesn't exist
Image building targets are dependencies of test running targets, so when
a docker image doesn't exist, it means it's skipped (due to dependency
checks in pre script). Therefore, skip the test too.
It is very easy to figure out current directory and bash option from the
execution, so do less in the Makefile invocation command line, and
figure both options in the script.
Alex Bennée [Tue, 19 Jul 2016 13:20:40 +0000 (21:20 +0800)]
tests/docker/docker.py: add update operation
This adds a new operation to the docker script to allow updating of
binaries in an existing container. This is because it would be
inefficient to re-build the whole container just for an update to the
QEMU binary.
Alex Bennée [Tue, 19 Jul 2016 13:20:39 +0000 (21:20 +0800)]
tests/docker/dockerfiles: new debian-bootstrap.docker
Together with the debian-bootstrap.pre script can now build an arbitrary
architecture of Debian using debootstrap. This allows debootstrap to set
up its first stage before the container is built.
To build a container you need a command line like:
Although a number of non-debian systems package the debootstrap script
it is fairly portable in itself. Assuming we have some sort of fakeroot
implementation we can just clone the upstream repository and use the
script from there.
Alex Bennée [Tue, 19 Jul 2016 13:20:38 +0000 (21:20 +0800)]
tests/docker/docker.py: check and run .pre script
The docker script will now search for an associated $dockerfile.pre
script which gets run in the same build context as the dockerfile will
be. This is to support pre-seeding the build context before running the
docker build.
Alex Bennée [Tue, 19 Jul 2016 13:20:37 +0000 (21:20 +0800)]
tests/docker/docker.py: support --include-executable
When passed the path to a binary we copy it and any linked libraries (if
it is dynamically linked) into the docker build context. These can then
be included by a dockerfile with the line:
# Copy all of context into container
ADD . /
This is mainly intended for setting up foreign architecture docker
images which use qemu-$arch to do cross-architecture linux-user
execution. It also relies on the host and guest file-system following
reasonable multi-arch layouts so the copied libraries don't clash with
the guest ones.
Alex Bennée [Tue, 19 Jul 2016 13:20:36 +0000 (21:20 +0800)]
tests/docker/docker.py: docker_dir outside build
Instead of letting the build_image create the temporary working dir we
move the creation to the build command. This is preparation for the
later patches where additional files can be added to the build context
before the build step is run.
We also ensure we remove the build context after we are done (mkdtemp
doesn't do this automatically for you).
* remotes/riku/tags/pull-linux-user-20160719-2:
linux-user: AArch64 has sync_file_range, not sync_file_range2
linux-user: Fix type for SIOCATMARK ioctl
linux-user: define missing sparc syscalls
linux-user: Fix terminal control ioctls
linux-user: Add some new blk ioctls
linux-user: Handle short lengths in host_to_target_sockaddr()
linux-user: Forget about synchronous signal once it is delivered
linux-user: Correct type for LOOP_GET_STATUS{,64} ioctls
linux-user: Correct type for BLKSSZGET
linux-user: Add loop control ioctls
linux-user: Check sigsetsize argument to syscalls
linux-user: add nested netlink types
linux-user: convert sockaddr_ll from host to target
linux-user: add fd_trans helper in do_recvfrom()
linux-user: fix netlink memory corruption
linux-user: fd_trans_*_data() returns the length
qxl: fix qxl_set_dirty call in qxl_dirty_one_surface
qxl_set_dirty() expects start and end as range specification.
qxl_dirty_one_surface passes 'size' instead of 'offset + size' as end
parameter. Fix that. Also use uint64_t everywhere while being at it.
Bug was added by "e25139b qxl: set only off-screen surfaces dirty instead
of the whole vram" and carried forward unnoticed by "5cdc402 qxl: fix
surface migration".
block/gluster: add support for multiple gluster servers
This patch adds a way to specify multiple volfile servers to the gluster
block backend of QEMU with tcp|rdma transport types and their port numbers.
Problem:
Currently VM Image on gluster volume is specified like this:
file=gluster[+tcp]://host[:port]/testvol/a.img
Say we have three hosts in a trusted pool with replica 3 volume in action.
When the host mentioned in the command above goes down for some reason,
the other two hosts are still available. But there's currently no way
to tell QEMU about them.
Solution:
New way of specifying VM Image on gluster volume with volfile servers:
(We still support old syntax to maintain backward compatibility)
type => transport type used to connect to gluster management daemon,
it can be tcp|unix
host => host address (hostname/ipv4/ipv6 addresses/socket path)
port => port number on which glusterd is listening.
socket => path to socket file
This patch gives a mechanism to provide all the server addresses, which are in
replica set, so in case host1 is down VM can still boot from any of the
active hosts.
This is equivalent to the backup-volfile-servers option supported by
mount.glusterfs (FUSE way of mounting gluster volume)
gluster volfile server fetch happens through unix and/or tcp, it doesn't
support volfile fetch over rdma. The rdma code may actually mislead,
so to make sure things do not break, for now we fallback to tcp when requested
for rdma, with a warning.
If you are wondering how this worked all these days, its the gluster libgfapi
code which handles anything other than unix transport as socket/tcp, sad but
true.
Also gluster doesn't support ipv6 addresses, removing the ipv6 related
comments/docs section
[Jeff: Minor grammatical fixes in comments and commit message, per
review comments]
A future patch will add support for multiple gluster servers. Existing
terminology is a bit unusual in relation to what names are used by
other networked devices, and doesn't map very well to the terminology
we expect to use for multiple servers. Therefore, rename the following
options:
'server' -> 'host'
'image' -> 'path'
'volname' -> 'volume'
Denis V. Lunev [Wed, 22 Jun 2016 12:35:27 +0000 (15:35 +0300)]
mirror: fix request throttling in drive-mirror
There are 2 deficiencies here:
- mirror_iteration could start several requests inside. Thus we could
simply have more in_flight requests than MAX_IN_FLIGHT.
- keeping this in mind throttling in mirror_run which is checking
s->in_flight == MAX_IN_FLIGHT is wrong.
The patch adds the check and throttling into mirror_iteration and fixes
the check in mirror_run() to be sure.
mirror: improve performance of mirroring of empty disk
We should not take into account zero blocks for delay calculations.
They are not read and thus IO throttling is not required. In the
other case VM migration with 16 Tb QCOW2 disk with 4 Gb of data takes
days.
With a bdrv_co_write_zeroes method on a target BDS and when this method
is working as indicated by the bdrv_can_write_zeroes_with_unmap(), zeroes
will not be placed into the wire. Thus the target could be very efficiently
zeroed out. This should be done with the largest chunk possible.
block: remove extra condition in bdrv_can_write_zeroes_with_unmap
All .bdrv_co_write_zeroes callbacks nowadays work perfectly even
with backing store attached. If future new callbacks would be unable to do
that - they have a chance to block this in bdrv_get_info().
Eric Blake [Thu, 14 Jul 2016 03:50:23 +0000 (21:50 -0600)]
qapi: Change Netdev into a flat union
This is a mostly-mechanical conversion that creates a new flat
union 'Netdev' QAPI type that covers all the branches of the
former 'NetClientOptions' simple union, where the branches are
now listed in a new 'NetClientDriver' enum rather than generated
from the simple union. The existence of a flat union has no
change to the command line syntax accepted for new code, and
will make it possible for a future patch to switch the QMP
command to parse a boxed union for no change to valid QMP; but
it does have some ripple effect on the C code when dealing with
the new types.
While making the conversion, note that the 'NetLegacy' type
remains unchanged: it applies only to legacy command line options,
and will not be ported to QMP, so it should remain a wrapper
around a simple union; to avoid confusion, the type named
'NetClientOptions' is now gone, and we introduce 'NetLegacyOptions'
in its place. Then, in the C code, we convert from NetLegacy to
Netdev as soon as possible, so that the bulk of the net stack
only has to deal with one QAPI type, not two. Note that since
the old legacy code always rejected 'hubport', we can just omit
that branch from the new 'NetLegacyOptions' simple union.
Based on an idea originally by Zoltán Kővágó <[email protected]>:
Message-Id: <01a527fbf1a5de880091f98cf011616a78adeeee.1441627176[email protected]>
although the sed script in that patch no longer applies due to
other changes in the tree since then, and I also did some manual
cleanups (such as fixing whitespace to keep checkpatch happy).
Peter Maydell [Mon, 11 Jul 2016 18:22:52 +0000 (19:22 +0100)]
arm_gicv3: Add assert()s to tell Coverity that offsets are aligned
Coverity complains that the GICR_IPRIORITYR case in gicv3_readl()
can overflow an array, because it doesn't know that the offsets
passed to that function must be word aligned. Add some assert()s
which hopefully tell Coverity that this isn't possible.
Peter Maydell [Mon, 11 Jul 2016 18:09:12 +0000 (19:09 +0100)]
target-arm: Fix unreachable code in gicv3_class_name()
Coverity complains that the exit() in gicv3_class_name()
can be unreachable, because if TARGET_AARCH64 is defined
then all code paths return before reaching it. Move the
exit() up to the error_report() that it belongs with.
Peter Maydell [Tue, 19 Jul 2016 14:04:36 +0000 (15:04 +0100)]
disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers
disas/bfd.h defines ATTRIBUTE_UNUSED, but unfortunately the
ALSA system headers also define this macro, which means that
you can get a compilation failure if building with ALSA and
any files happen to include the alsa headers before bfd.h
rather than the other way around.
This is unfortunate namespace pollution by the ALSA headers but
we can work around it. Add an #ifndef guard to bfd.h and remove
the unnecessary extra definition in disas/arm.c to fix this.
Peter Maydell [Tue, 19 Jul 2016 14:08:05 +0000 (15:08 +0100)]
Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging
* two old patches from prospective GSoC students
* i386 -kernel device tree support
* Coverity fix
* memory usage improvement from Peter
* checkpatch fix
* g_path_get_dirname cleanup
* caching of block status for iSCSI
* remotes/bonzini/tags/for-upstream:
target-i386: Remove redundant HF_SOFTMMU_MASK
block/iscsi: allow caching of the allocation map
block/iscsi: fix rounding in iscsi_allocationmap_set
Move README to markdown
cpu-exec: Move down some declarations in cpu_exec()
exec: avoid realloc in phys_map_node_reserve
checkpatch: consider git extended headers valid patches
megasas: remove useless check for cmd->frame
compiler: never omit assertions if using a static analysis tool
hw/i386: add device tree support
Changed malloc to g_malloc, free to g_free in bsd-user/qemu.h
use g_path_get_dirname instead of dirname
Peter Maydell [Fri, 15 Jul 2016 16:28:06 +0000 (17:28 +0100)]
linux-user: AArch64 has sync_file_range, not sync_file_range2
The AArch64 Linux ABI syscall 84 is sync_file_range, not
sync_file_range2 (in the kernel it uses the asm-generic
headers and does not define __ARCH_WANT_SYNC_FILE_RANGE2).
Update our TARGET_NR_* definitions accordingly.
This fixes the sync_file_range syscall which otherwise
gets its arguments in the wrong order.
Peter Maydell [Fri, 15 Jul 2016 11:09:31 +0000 (12:09 +0100)]
linux-user: Fix type for SIOCATMARK ioctl
The SIOCATMARK ioctl takes an argument which should be a
pointer to an integer where the kernel will write the result.
We were incorrectly declaring it as TYPE_NULL which would mean
it would always fail (with EFAULT) when it should succeed.
Correct the type.
Timothy Pearson [Sun, 19 Jun 2016 00:15:35 +0000 (19:15 -0500)]
linux-user: Fix terminal control ioctls
TIOCGPTN and related terminal control ioctls were not converted to the guest ioctl format on x86_64 targets. Convert these ioctls to enable terminal functionality on x86_64 guests.
Peter Maydell [Thu, 7 Jul 2016 14:44:43 +0000 (15:44 +0100)]
linux-user: Handle short lengths in host_to_target_sockaddr()
If userspace specifies a short buffer for a target sockaddr,
the kernel will only copy in as much as it has space for
(or none at all if the length is zero) -- see the kernel
move_addr_to_user() function. Mimic this in QEMU's
host_to_target_sockaddr() routine.
In particular, this fixes a segfault running the LTP
recvfrom01 test, where the guest makes a recvfrom()
call with a bad buffer pointer and other parameters which
cause the kernel to set the addrlen to zero; because we
did not skip the attempt to swap the sa_family field we
segfaulted on the bad address.
Peter Maydell [Wed, 6 Jul 2016 14:09:29 +0000 (15:09 +0100)]
linux-user: Forget about synchronous signal once it is delivered
Commit 655ed67c2a248cf which switched synchronous signals to
benig recorded in ts->sync_signal rather than in a queue
with every other signal had a bug: we failed to clear
the flag indicating that a synchronous signal was pending
when we delivered it. This meant that we would take the signal
again and again every time the guest made a syscall.
(This is a bug introduced in my refactoring of Timothy Baldwin's
original code.)
Fix this by passing in the struct emulated_sigtable* to
handle_pending_signal(), so that we clear the pending flag
in the ts->sync_signal struct when handling a synchronous signal.
Peter Maydell [Tue, 5 Jul 2016 15:36:29 +0000 (16:36 +0100)]
linux-user: Correct type for LOOP_GET_STATUS{,64} ioctls
The LOOP_GET_STATUS and LOOP_GET_STATUS64 ioctls were incorrectly
defined as IOC_W rather than IOC_R, which meant we weren't
correctly copying the information back from the kernel to the guest.
The loop_info64 structure definition was also missing a member
and using the wrong type for several 32-bit fields.
In particular, this meant that "kpartx -d image.img" didn't work
and "losetup -a" behaved strangely. Correct the ioctl type definitions.
Peter Maydell [Mon, 4 Jul 2016 16:06:18 +0000 (17:06 +0100)]
linux-user: Correct type for BLKSSZGET
The BLKSSZGET ioctl takes an argument which is a pointer to an int.
We were incorrectly declaring it to take a pointer to a long, which
meant that we would incorrectly write to memory which we should not
if the guest is a 64-bit architecture.
In particular, kpartx uses this ioctl to write to an int on the
stack, which tends to result in it crashing immediately.
Peter Maydell [Thu, 30 Jun 2016 13:23:24 +0000 (14:23 +0100)]
linux-user: Check sigsetsize argument to syscalls
Many syscalls which take a sigset_t argument also take an argument
giving the size of the sigset_t. The kernel insists that this
matches its idea of the type size and fails EINVAL if it is not.
Implement this logic in QEMU. (This mostly just means some LTP test
cases which check error cases now pass.)
Laurent Vivier [Mon, 27 Jun 2016 16:54:30 +0000 (18:54 +0200)]
linux-user: add nested netlink types
Nested types are used by the kernel to send link information and
protocol properties.
We can see following errors with "ip link show":
Unimplemented nested type 26
Unimplemented nested type 26
Unimplemented nested type 18
Unimplemented nested type 26
Unimplemented nested type 18
Unimplemented nested type 26
This patch implements nested types 18 (IFLA_LINKINFO) and
26 (IFLA_AF_SPEC).
Laurent Vivier [Tue, 21 Jun 2016 17:51:14 +0000 (19:51 +0200)]
linux-user: fix netlink memory corruption
Netlink is byte-swapping data in the guest memory (it's bad).
It's ok when the data come from the host as they are generated by the
host.
But it doesn't work when data come from the guest: the guest can
try to reuse these data whereas they have been byte-swapped.
This is what happens in glibc:
glibc generates a sequence number in nlh.nlmsg_seq and calls
sendto() with this nlh. In sendto(), we byte-swap nlmsg.seq.
Later, after the recvmsg(), glibc compares nlh.nlmsg_seq with
sequence number given in return, and of course it fails (hangs),
because nlh.nlmsg_seq is not valid anymore.
The involved code in glibc is:
sysdeps/unix/sysv/linux/check_pf.c:make_request()
...
req.nlh.nlmsg_seq = time (NULL);
...
if (TEMP_FAILURE_RETRY (__sendto (fd, (void *) &req, sizeof (req), 0,
(struct sockaddr *) &nladdr,
sizeof (nladdr))) < 0)
<here req.nlh.nlmsg_seq has been byte-swapped>
...
do
{
...
ssize_t read_len = TEMP_FAILURE_RETRY (__recvmsg (fd, &msg, 0));
...
struct nlmsghdr *nlmh;
for (nlmh = (struct nlmsghdr *) buf;
NLMSG_OK (nlmh, (size_t) read_len);
nlmh = (struct nlmsghdr *) NLMSG_NEXT (nlmh, read_len))
{
<we compare nlmh->nlmsg_seq with corrupted req.nlh.nlmsg_seq>
if (nladdr.nl_pid != 0 || (pid_t) nlmh->nlmsg_pid != pid
|| nlmh->nlmsg_seq != req.nlh.nlmsg_seq)
continue;
...
else if (nlmh->nlmsg_type == NLMSG_DONE)
/* We found the end, leave the loop. */
done = true;
}
}
while (! done);
As we have a continue on "nlmh->nlmsg_seq != req.nlh.nlmsg_seq",
"done" cannot be set to "true" and we have an infinite loop.
It's why commands like "apt-get update" or "dnf update hangs".
Peter Maydell [Tue, 19 Jul 2016 12:00:35 +0000 (13:00 +0100)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
# gpg: Signature made Tue 19 Jul 2016 03:33:40 BST
# gpg: using RSA key 0xEF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <[email protected]>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* remotes/jasowang/tags/net-pull-request:
e1000e: fix building without CONFIG_VMXNET3_PCI
MAINTAINERS: release Scott from being a rocker maintainer
tap: fix memory leak on failure to create a multiqueue tap device
net: fix incorrect argument to iov_to_buf
net: fix incorrect access to pointer
e1000e: fix incorrect access to pointer
Eric Blake [Thu, 14 Jul 2016 22:37:58 +0000 (16:37 -0600)]
block: Simplify drive-mirror
Now that we can support boxed commands, use it to greatly
reduce the number of parameters (and likelihood of getting
out of sync) when adjusting drive-mirror parameters.
Eric Blake [Thu, 14 Jul 2016 03:50:21 +0000 (21:50 -0600)]
block: Simplify block_set_io_throttle
Now that we can support boxed commands, use it to greatly
reduce the number of parameters (and likelihood of getting
out of sync) when adjusting throttle parameters.
Eric Blake [Thu, 14 Jul 2016 03:50:20 +0000 (21:50 -0600)]
qapi: Implement boxed types for commands/events
Turn on the ability to pass command and event arguments in
a single boxed parameter, which must name a non-empty type
(although the type can be a struct with all optional members).
For structs, it makes it possible to pass a single qapi type
instead of a breakout of all struct members (useful if the
arguments are already in a struct or if the number of members
is large); for other complex types, it is now possible to use
a union or alternate as the data for a command or event.
The empty type may be technically feasible if needed down the
road, but it's easier to forbid it now and relax things to allow
it later, than it is to allow it now and have to special case
how the generated 'q_empty' type is handled (see commit 7ce106a9
for reasons why nothing is generated for the empty type). An
alternate type is never considered empty, but now that a boxed
type can be either an object or an alternate, we have to provide
a trivial QAPISchemaAlternateType.is_empty(). The new call to
arg_type.is_empty() during QAPISchemaCommand.check() requires
that we first check the type in question; but there is no chance
of introducing a cycle since objects do not refer back to commands.
We still have a split in syntax checking between ad-hoc parsing
up front (merely validates that 'boxed' has a sane value) and
during .check() methods (if 'boxed' is set, then 'data' must name
a non-empty user-defined type).
Generated code is unchanged, as long as no client uses the
new feature.
Eric Blake [Thu, 14 Jul 2016 03:50:19 +0000 (21:50 -0600)]
qapi: Plumb in 'boxed' to qapi generator lower levels
The next patch will add support for passing a qapi union type
as the 'data' of a command. But to do that, the user function
for implementing the command, as called by the generated
marshal command, must take the corresponding C struct as a
single boxed pointer, rather than a breakdown into one
parameter per member. Even without a union, being able to use
a C struct rather than a list of parameters can make it much
easier to handle coding with QAPI.
This patch adds the internal plumbing of a 'boxed' flag
associated with each command and event. In several cases,
this means adding indentation, with one new dead branch and
the remaining branch being the original code more deeply
nested; this was done so that the new implementation in the
next patch is easier to review without also being mixed with
indentation changes.
For this patch, no behavior or generated output changes, other
than the testsuite outputting the value of the new flag
(always False for now).
Eric Blake [Thu, 14 Jul 2016 03:50:18 +0000 (21:50 -0600)]
qapi-event: Simplify visit of non-implicit data
Commit 7ce106a9 documented why we don't generated a visit_type_FOO()
for implicit types; and therefore events with an anonymous type for
'data' have to open-code a visit. Note that the open-coded visit in
qapi-event.c is slightly different from what is done in
qapi-visit.c for normal types, in part because we don't have to
check for *obj being NULL or free things on error. But where the
type is not implicit, it is nicer to reuse the normal visit instead
of open-coding a duplicate.
At the moment, the only event with a non-implicit 'data' is in the
testsuite, where test-qapi-event.c changes as follows:
Eric Blake [Thu, 14 Jul 2016 03:50:17 +0000 (21:50 -0600)]
qapi: Drop useless gen_err_check()
Ever since commit 12f254f removed the last parameterization
of gen_err_check(), it no longer makes sense to hide the three
lines of generated C code behind a macro call. Just inline it
into the remaining users.
Eric Blake [Thu, 14 Jul 2016 03:50:16 +0000 (21:50 -0600)]
qapi: Add type.is_empty() helper
In the near future, we want to lift our artificial restriction of
no variants at the top level of an event, at which point the
currently open-coded check for empty members will become
insufficient. Factor it out into a new helper method is_empty()
now, and future-proof it by checking variants, too, along with an
assert that it is not used prior to the completion of .check().
Update places that were checking for (non-)empty .members to use
the new helper.
All of the current callers assert that there are no variants (either
directly, or by qapi.py asserting that base types have no variants),
so this is not a semantic change.
Eric Blake [Thu, 14 Jul 2016 03:50:15 +0000 (21:50 -0600)]
qapi: Hide tag_name data member of variants
Clean up the only remaining external use of the tag_name field of
QAPISchemaObjectTypeVariants, by explicitly listing the generated
'type' tag for all variants in the testsuite (you can still tell
simple unions by the -wrapper types). Then we can mark the
tag_name field as private by adding a leading underscore to prevent
any further use.
Eric Blake [Thu, 14 Jul 2016 03:50:14 +0000 (21:50 -0600)]
qapi: Special case c_name() for empty type
Commit 7ce106a rendered QAPISchemaObjectType.c_name() redundant,
since it now does nothing more than delegate to its superclass.
However, rather than deleting it, we can restore part of the
assertion that was removed in that commit, to prove that we never
emit the empty type directly in generated code, but rather
special-case it as a built-in that makes other aspects of code
generation easier to reason about.
Eric Blake [Thu, 14 Jul 2016 03:50:13 +0000 (21:50 -0600)]
qapi: Require all branches of flat union enum to be covered
We were previously enforcing that all flat union branches were
found in the corresponding enum, but not that all enum values
were covered by branches. The resulting generated code would
abort() if the user passes the uncovered enum value.
We don't automatically treat non-present branches in a flat
union as empty types, for symmetry with simple unions (there,
the enum type is generated from the list of all branches, so
there is no way to omit a branch but still have it be part of
the union).
A later patch will add shorthand so that branches that are empty
in flat unions can be declared as 'branch':{} instead of
'branch':'Empty', to avoid the need for an otherwise useless
explicit empty type. [Such shorthand for simple unions is a bit
harder to justify, since we would still have to generate a
wrapper type that parses 'data':{}, rather than truly being an
empty branch with no additional siblings to the 'type' member.]
[rework net_client_init1() to pass Netdev by copying from NetdevLegacy,
rather than merging the two types - which means that we still need
NetClientOptions after all. Rebase to qapi changes. The bulk of the
patch is mechanical, replacing 'opts' by 'netdev->opts', while
net_client_init1() takes care of converting between legacy and modern
types.]
Paolo Bonzini [Thu, 7 Jul 2016 15:53:18 +0000 (17:53 +0200)]
qapi: change QmpInputVisitor to QSLIST
This saves a lot of memory compared to a statically-sized array,
or at least 24kb could be considered a lot on an Atari ST.
It also makes the code more similar to QmpOutputVisitor.
This removes the limit on the depth of a QObject that can be processed
into a QAPI tree. This is not a problem because QObjects can be
considered trusted; the text received on the QMP wire is untrusted
input, but the JSON parser already takes pains to limit the QObject tree
it creates. We don't need the QMP input visitor to limit it again.
* remotes/jnsnow/tags/ide-pull-request:
block: ignore flush requests when storage is clean
tests: in IDE and AHCI tests perform DMA write before flushing
ide: set retry_unit for PIO and FLUSH requests
ide: refactor retry_unit set and clear into separate function
* remotes/stefanha/tags/tracing-pull-request:
trace: Add QAPI/QMP interfaces to query and control per-vCPU tracing state
trace: Allow event name pattern in "info trace-events"
trace: Conditionally trace events based on their per-vCPU state
trace: Add per-vCPU tracing states for events with the 'vcpu' property
trace: Cosmetic changes on fast-path tracing
disas: Remove unused macro '_'
trace: Identify events with the 'vcpu' property
trace: [bsd-user] Commandline arguments to control tracing
trace: [linux-user] Commandline arguments to control tracing
Peter Maydell [Tue, 19 Jul 2016 08:02:05 +0000 (09:02 +0100)]
Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20160718.0' into staging
VFIO update 2016-07-18
One fix for 2.7-rc0 which hides the ARI extended capability, fixing
multifunction support in PCIe configurations where the assigned device
function topology does not match the host (Alex Williamson)
Peter Lieven [Mon, 18 Jul 2016 08:52:20 +0000 (10:52 +0200)]
block/iscsi: allow caching of the allocation map
until now the allocation map was used only as a hint if a cluster
is allocated or not. If a block was not allocated (or Qemu had
no info about the allocation status) a get_block_status call was
issued to check the allocation status and possibly avoid
a subsequent read of unallocated sectors. If a block known to be
allocated the get_block_status call was omitted. In the other case
a get_block_status call was issued before every read to avoid
the necessity for a consistent allocation map. To avoid the
potential overhead of calling get_block_status for each and
every read request this took only place for the bigger requests.
This patch enhances this mechanism to cache the allocation
status and avoid calling get_block_status for blocks where
the allocation status has been queried before. This allows
for bypassing the read request even for smaller requests and
additionally omits calling get_block_status for known to be
unallocated blocks.
Move the README file to markdown so that it makes the github page look
prettier. I know that github repo is a mirror and not the official
repo, but I think it doesn't hurt to have it in markdown format.
block: ignore flush requests when storage is clean
Some guests (win2008 server for example) do a lot of unnecessary
flushing when underlying media has not changed. This adds additional
overhead on host when calling fsync/fdatasync.
This change introduces a write generation scheme in BlockDriverState.
Current write generation is checked against last flushed generation to
avoid unnessesary flushes.
The problem with excessive flushing was found by a performance test
which does parallel directory tree creation (from 2 processes).
Results improved from 0.424 loops/sec to 0.432 loops/sec.
Each loop creates 10^3 directories with 10 files in each.
This affected some blkdebug testcases that were expecting error logs from
failure-injected flushes which are now skipped entirely
(tests 026 071 089).
This also affects the performance of block jobs and thus BLOCK_JOB_READY
events for driver-mirror and active block-commit commands now arrives
faster, before QMP send successfully returns to caller (tests 141 144).
The following sequence of tests discovered a problem in IDE emulation:
1. Send DMA write to IDE device 0
2. Send CMD_FLUSH_CACHE to same IDE device which will be failed by block
layer using blkdebug script in tests/ide-test:test_retry_flush
When doing DMA request ide/core.c will set s->retry_unit to s->unit in
ide_start_dma. When dma completes ide_set_inactive sets retry_unit to -1.
After that ide_flush_cache runs and fails thanks to blkdebug.
ide_flush_cb calls ide_handle_rw_error which asserts that s->retry_unit
== s->unit. But s->retry_unit is still -1 after previous DMA completion
and flush does not use anything related to retry.
This patch restricts retry unit assertion only to ops that actually use
retry logic.
trace: Conditionally trace events based on their per-vCPU state
Events with the 'vcpu' property are conditionally emitted according to
their per-vCPU state. Other events are emitted normally based on their
global tracing state.
Note that the per-vCPU condition check applies to all tracing backends.
Eliminates a future compilation error when UI code includes the tracing
headers (indirectly pulling "disas/bfd.h" through "qom/cpu.h") and
GLib's i18n '_' macro.
trace: [bsd-user] Commandline arguments to control tracing
[Changed const char *trace_file to char *trace_file since it's a
heap-allocated string that needs to be freed. This type is also
returned by trace_opt_parse() and used in vl.c.
Also fixed coding style on for(;;) and else statement as suggested by
Eric Blake <[email protected]> since the patch modifies these lines or
close enough.
--Stefan]
trace: [linux-user] Commandline arguments to control tracing
[Changed const char *trace_file to char *trace_file since it's a
heap-allocated string that needs to be freed. This type is also
returned by trace_opt_parse() and used in vl.c.
--Stefan]
Alex Williamson [Mon, 18 Jul 2016 16:55:17 +0000 (10:55 -0600)]
vfio/pci: Hide ARI capability
QEMU supports ARI on downstream ports and assigned devices may support
ARI in their extended capabilities. The endpoint ARI capability
specifies the next function, such that the OS doesn't need to walk
each possible function, however this next function is relative to the
host, not the guest. This leads to device discovery issues when we
combine separate functions into virtual multi-function packages in a
guest. For example, SR-IOV VFs are not enumerated by simply probing
the function address space, therefore the ARI next-function field is
zero. When we combine multiple VFs together as a multi-function
device in the guest, the guest OS identifies ARI is enabled, relies on
this next-function field, and stops looking for additional function
after the first is found.
Long term we should expose the ARI capability to the guest to enable
configurations with more than 8 functions per slot, but this requires
additional QEMU PCI infrastructure to manage the next-function field
for multiple, otherwise independent devices. In the short term,
hiding this capability allows equivalent functionality to what we
currently have on non-express chipsets.
Pranith Kumar [Mon, 27 Jun 2016 18:13:22 +0000 (14:13 -0400)]
.travis.yml: Disable IRC build status updates from forks
We want the travis build bot to post notifications on IRC only for the
master qemu repository and not the various forks/branches of
others. Currently there is no direct option to restrict the updates to
one repository. This is being worked upon by the developers and
tracked in https://github.com/travis-ci/travis-ci/issues/1094.
Until such time, we can use the workaround as posted in
ref. https://github.com/facebook/flow/pull/1822.
This basically creates an ecrypted string which decrypts to qemu IRC
channel only on "qemu/qemu" repo and not on the forks. This enables
the build bot to notify the IRC only for the main repo.
Renames look like this with git-diff(1) when diff.renames = true is set:
diff --git a/a b/b
similarity index 100%
rename from a
rename to b
This raises the "Does not appear to be a unified-diff format patch"
error because checkpatch.pl only considers a diff valid if it contains
at least one "@@" hunk.
This patch accepts renames and copies too so that checkpatch.pl exits
successfully when a diff only renames/copies files. The git diff
extended header format is described on the git-diff(1) man page.
Roman Pen [Wed, 13 Jul 2016 13:03:24 +0000 (15:03 +0200)]
linux-aio: prevent submitting more than MAX_EVENTS
Invoking io_setup(MAX_EVENTS) we ask kernel to create ring buffer for us
with specified number of events. But kernel ring buffer allocation logic
is a bit tricky (ring buffer is page size aligned + some percpu allocation
are required) so eventually more than requested events number is allocated.
From a userspace side we have to follow the convention and should not try
to io_submit() more or logic, which consumes completed events, should be
changed accordingly. The pitfall is in the following sequence:
MAX_EVENTS = 128
io_setup(MAX_EVENTS)
io_submit(MAX_EVENTS)
io_submit(MAX_EVENTS)
/* now 256 events are in-flight */
io_getevents(MAX_EVENTS) = 128
/* we can handle only 128 events at once, to be sure
* that nothing is pended the io_getevents(MAX_EVENTS)
* call must be invoked once more or hang will happen. */
To prevent the hang or reiteration of io_getevents() call this patch
restricts the number of in-flights, which is now limited to MAX_EVENTS.
projects / qemu.git / log