Alex Bennée [Fri, 9 Aug 2019 13:50:52 +0000 (14:50 +0100)]
tests/docker: set DEF_TARGET_LIST for some containers
You can assume the failures most people are interested in are the
cross-compile failures that are specific to the cross compile target.
Set DEF_TARGET_LIST based on what we use for shippable, the user can
always override by calling with TARGET_LIST set.
Paolo Bonzini [Wed, 7 Aug 2019 14:35:23 +0000 (16:35 +0200)]
tests/tcg: move configuration to a sub-shell script
Avoid the repeated inclusions of config-target.mak, which have
risks of namespace pollution, and instead build minimal configuration
files in a configuration script. The same configuration files can
also be included in Makefile and Makefile.qemu
[AJB 10/09/19]
In the original PR this had inadvertently enabled tests
for ppc64abi32. However as the rest of the multiarch tests work rather
than disabling the otherwise correctly functioning build I've just
skipped the failing linux-test test. For some reason I can't debug it
with TCG so I'm leaving that to the PPC maintainers to look at.
Paolo Bonzini [Wed, 7 Aug 2019 14:35:22 +0000 (16:35 +0200)]
tests/tcg: cleanup Makefile inclusions
Rename Makefile.probe to Makefile.prereqs and make it actually
define rules for the tests.
Rename Makefile to Makefile.target, since it is not a toplevel
makefile.
Rename Makefile.include to Makefile.qemu and disentangle it
from the QEMU Makefile.target, so that it is invoked recursively
by tests/Makefile.include. Tests are now placed in
tests/tcg/$(TARGET).
Drop the usage of TARGET_BASE_ARCH, which is ignored by everything except
x86_64 and aarch64. Fix x86 tests by using -cpu max and, while
at it, standardize on QEMU_OPTS for aarch64 tests too.
Paolo Bonzini [Wed, 7 Aug 2019 14:35:21 +0000 (16:35 +0200)]
tests/tcg: use EXTRA_CFLAGS everywhere
For i386 specifically, this allows using the host GCC
to compile the i386 tests. But, it should really be
done for all targets, unless we want to pass $(EXTRA_CFLAGS)
directly as part of $(CC).
Alex Bennée [Wed, 4 Sep 2019 17:46:36 +0000 (18:46 +0100)]
tests/docker: handle missing encoding keyword for subprocess.check_output
This was only added in Python 3.6 and not all the build hosts have
that recent a python3. However we still need to ensure everything is
returns as a unicode string so checks higher up the call chain don't
barf.
Signed-off-by: Alex Bennée <[email protected]>
fixup! tests/docker: handle missing encoding keyword for subprocess.check_output
Alex Bennée [Wed, 4 Sep 2019 09:07:17 +0000 (10:07 +0100)]
tests/docker: fix "cc" command to work with podman
Podman requires a little bit of additional magic to the uid mapping
which was already done for the normal RunCommand. We simplify the
logic by pushing it directly into the Docker::run method to avoid
instantiating an extra Docker() object and ensure the CC command
always runs as the current user.
John Snow [Wed, 4 Sep 2019 23:24:51 +0000 (19:24 -0400)]
tests/docker: Use --userns=keep-id for podman
The workaround that attempts to accomplish the same result as --userns=keep-id
does not appear to work well with UIDs much above 1000 (like mine, which is
above 20000.)
Since we have official support for this "trick" now, use the supported method.
The introduction of podman support inadvertently broke configure's
detect of the container support as the configure probe didn't specify
an engine type. To fix this in docker.py:
- only (re)set USE_ENGINE if --engine is specified
- enhance the output so docker is no longer just yes
In the configure script we can at least start cleaning up the
detecting and naming of variables. To avoid too much churn the
conversion of the various make DOCKER_foo variables has been left for
future clean-ups.
Thomas Huth [Mon, 2 Sep 2019 16:26:38 +0000 (18:26 +0200)]
hw/misc: Mark most objects as "common" code to speed up compilation a litte bit
Most of the code in hw/misc/ does not directly depend on CPU-specific
code. Mark it as "common" so that the code can be shared between e.g.
qemu-system-arm and qemu-system-aarch64, or between the various mips
flavours, instead of recompiling it for each and every target again
and again.
* remotes/huth-gitlab/tags/m68k-pull-2019-09-07:
.travis.yml: Let the avocado job run the NeXTcube tests
tests/acceptance: Add test of NeXTcube framebuffer using OCR
m68k: Add an entry for the NeXTcube machine to the MAINTAINERS file
m68k: Add serial controller to the NeXTcube machine
escc: introduce a selector for the register bit
m68k: Add NeXTcube machine
m68k: Add NeXTcube keyboard device
m68k: Add NeXTcube framebuffer device emulation
tests/acceptance: Add test of NeXTcube framebuffer using OCR
Add a test of the NeXTcube framebuffer using the Tesseract OCR
engine on a screenshot of the framebuffer device.
The test is very quick:
$ avocado --show=app,console run tests/acceptance/machine_m68k_nextcube.py
JOB ID : 78844a92424cc495bd068c3874d542d1e20f24bc
JOB LOG : /home/phil/avocado/job-results/job-2019-08-13T13.16-78844a9/job.log
(1/3) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_size: PASS (2.16 s)
(2/3) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v3: -
ue r pun Honl'flx ; 5‘ 55‘
avg ncaaaaa 25 MHZ, memary jag m
Backplane slat «a
Ethernet address a a r a r3 2
Memgry sackets aea canflqured far 16MB Darlly page made stMs but have 16MB page made stMs )nstalled
Memgry sackets a and 1 canflqured far 16MB Darlly page made stMs but have 16MB page made stMs )nstalled
[...]
Yestlnq the rpu, 5::
system test raneg Errar egge 51
Egg: cammand
Default pggc devlce nut fauna
NEXY>I
PASS (2.64 s)
(3/3) tests/acceptance/machine_m68k_nextcube.py:NextCubeMachine.test_bootrom_framebuffer_ocr_with_tesseract_v4: SKIP: tesseract v4 OCR tool not available
RESULTS : PASS 2 | ERROR 0 | FAIL 0 | SKIP 1 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 5.35 s
Documentation on how to install tesseract:
https://github.com/tesseract-ocr/tesseract/wiki#installation
Thomas Huth [Sat, 30 Jun 2018 08:33:57 +0000 (08:33 +0000)]
m68k: Add an entry for the NeXTcube machine to the MAINTAINERS file
I don't have much clue about the NeXT hardware, but at least I know now
the source files a little bit, so I volunteer to pick up patches and send
PULL requests for them until someone else with more knowledge steps up
to do this job instead.
Thomas Huth [Sat, 30 Jun 2018 14:58:01 +0000 (16:58 +0200)]
m68k: Add serial controller to the NeXTcube machine
The NeXTcube uses a normal 8530 serial controller, so we can simply use
our normal "escc" device here.
While we're at it, also add a boot-serial-test for the next-cube machine,
now that the serial output works.
Thomas Huth [Sat, 30 Jun 2018 06:45:25 +0000 (08:45 +0200)]
m68k: Add NeXTcube machine
It is still quite incomplete (no SCSI, no floppy emulation, no network,
etc.), but the firmware already shows up the debug monitor prompt in the
framebuffer display, so at least the very basics are already working.
This code has been taken from Bryce Lanham's GSoC 2011 NeXT branch at
Thomas Huth [Sat, 30 Jun 2018 05:50:23 +0000 (07:50 +0200)]
m68k: Add NeXTcube keyboard device
It is likely still quite incomplete (e.g. mouse and interrupts are not
implemented yet), but it is good enough for keyboard input at the firmware
monitor.
This code has been taken from Bryce Lanham's GSoC 2011 NeXT branch at
Thomas Huth [Sat, 30 Jun 2018 05:19:42 +0000 (07:19 +0200)]
m68k: Add NeXTcube framebuffer device emulation
The NeXTcube uses a linear framebuffer with 4 greyscale colors and
a fixed resolution of 1120 * 832.
This code has been taken from Bryce Lanham's GSoC 2011 NeXT branch at
Peter Maydell [Fri, 6 Sep 2019 08:28:31 +0000 (09:28 +0100)]
Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2019-09-05-v2' into staging
nbd patches for 2019-09-05
- Advertise NBD_FLAG_CAN_MULTI_CONN on readonly images
- Tolerate larger set of server error responses during handshake
- More precision on handling fallocate() failures due to alignment
- Better documentation of NBD connection URIs
- Implement new extension NBD_CMD_FLAG_FAST_ZERO to benefit qemu-img convert
* remotes/ericb/tags/pull-nbd-2019-09-05-v2:
nbd: Implement server use of NBD FAST_ZERO
nbd: Implement client use of NBD FAST_ZERO
nbd: Prepare for NBD_CMD_FLAG_FAST_ZERO
nbd: Improve per-export flag handling in server
docs: Update preferred NBD device syntax
block: workaround for unaligned byte range in fallocate()
nbd: Tolerate more errors to structured reply request
nbd: Use g_autofree in a few places
nbd: Advertise multi-conn for shared read-only connections
Eric Blake [Fri, 23 Aug 2019 14:37:25 +0000 (09:37 -0500)]
nbd: Implement server use of NBD FAST_ZERO
The server side is fairly straightforward: we can always advertise
support for detection of fast zero, and implement it by mapping the
request to the block layer BDRV_REQ_NO_FALLBACK.
Eric Blake [Fri, 23 Aug 2019 14:37:24 +0000 (09:37 -0500)]
nbd: Implement client use of NBD FAST_ZERO
The client side is fairly straightforward: if the server advertised
fast zero support, then we can map that to BDRV_REQ_NO_FALLBACK
support. A server that advertises FAST_ZERO but not WRITE_ZEROES
is technically broken, but we can ignore that situation as it does
not change our behavior.
Eric Blake [Fri, 23 Aug 2019 14:37:23 +0000 (09:37 -0500)]
nbd: Prepare for NBD_CMD_FLAG_FAST_ZERO
Commit fe0480d6 and friends added BDRV_REQ_NO_FALLBACK as a way to
avoid wasting time on a preliminary write-zero request that will later
be rewritten by actual data, if it is known that the write-zero
request will use a slow fallback; but in doing so, could not optimize
for NBD. The NBD specification is now considering an extension that
will allow passing on those semantics; this patch updates the new
protocol bits and 'qemu-nbd --list' output to recognize the bit, as
well as the new errno value possible when using the new flag; while
upcoming patches will improve the client to use the feature when
present, and the server to advertise support for it.
The NBD spec recommends (but not requires) that ENOTSUP be avoided for
all but failures of a fast zero (the only time it is mandatory to
avoid an ENOTSUP failure is when fast zero is supported but not
requested during write zeroes; the questionable use is for ENOTSUP to
other actions like a normal write request). However, clients that get
an unexpected ENOTSUP will either already be treating it the same as
EINVAL, or may appreciate the extra bit of information. We were
equally loose for returning EOVERFLOW in more situations than
recommended by the spec, so if it turns out to be a problem in
practice, a later patch can tighten handling for both error codes.
Eric Blake [Fri, 23 Aug 2019 14:37:22 +0000 (09:37 -0500)]
nbd: Improve per-export flag handling in server
When creating a read-only image, we are still advertising support for
TRIM and WRITE_ZEROES to the client, even though the client should not
be issuing those commands. But seeing this requires looking across
multiple functions:
All callers to nbd_export_new() passed a single flag based solely on
whether the export allows writes. Later, we then pass a constant set
of flags to nbd_negotiate_options() (namely, the set of flags which we
always support, at least for writable images), which is then further
dynamically modified with NBD_FLAG_SEND_DF based on client requests
for structured options. Finally, when processing NBD_OPT_EXPORT_NAME
or NBD_OPT_EXPORT_GO we bitwise-or the original caller's flag with the
runtime set of flags we've built up over several functions.
Let's refactor things to instead compute a baseline of flags as soon
as possible which gets shared between multiple clients, in
nbd_export_new(), and changing the signature for the callers to pass
in a simpler bool rather than having to figure out flags. We can then
get rid of the 'myflags' parameter to various functions, and instead
refer to client for everything we need (we still have to perform a
bitwise-OR for NBD_FLAG_SEND_DF during NBD_OPT_EXPORT_NAME and
NBD_OPT_EXPORT_GO, but it's easier to see what is being computed).
This lets us quit advertising senseless flags for read-only images, as
well as making the next patch for exposing FAST_ZERO support easier to
write.
block: workaround for unaligned byte range in fallocate()
Revert the commit 118f99442d 'block/io.c: fix for the allocation failure'
and use better error handling for file systems that do not support
fallocate() for an unaligned byte range. Allow falling back to pwrite
in case fallocate() returns EINVAL.
Eric Blake [Sat, 24 Aug 2019 17:28:13 +0000 (12:28 -0500)]
nbd: Tolerate more errors to structured reply request
A server may have a reason to reject a request for structured replies,
beyond just not recognizing them as a valid request; similarly, it may
have a reason for rejecting a request for a meta context. It doesn't
hurt us to continue talking to such a server; otherwise 'qemu-nbd
--list' of such a server fails to display all available details about
the export.
Encountered when temporarily tweaking nbdkit to reply with
NBD_REP_ERR_POLICY. Present since structured reply support was first
added (commit d795299b reused starttls handling, but starttls is
different in that we can't fall back to other behavior on any error).
Note that for an unencrypted client trying to connect to a server that
requires encryption, this defers the point of failure to when we
finally execute a strict command (such as NBD_OPT_GO or NBD_OPT_LIST),
now that the intermediate NBD_OPT_STRUCTURED_REPLY does not diagnose
NBD_REP_ERR_TLS_REQD as fatal; but as the protocol eventually gets us
to a command where we can't continue onwards, the changed error
message doesn't cause any security concerns.
Eric Blake [Sat, 24 Aug 2019 17:28:12 +0000 (12:28 -0500)]
nbd: Use g_autofree in a few places
Thanks to our recent move to use glib's g_autofree, I can join the
bandwagon. Getting rid of gotos is fun ;)
There are probably more places where we could register cleanup
functions and get rid of more gotos; this patch just focuses on the
labels that existed merely to call g_free.
Eric Blake [Thu, 15 Aug 2019 18:50:24 +0000 (13:50 -0500)]
nbd: Advertise multi-conn for shared read-only connections
The NBD specification defines NBD_FLAG_CAN_MULTI_CONN, which can be
advertised when the server promises cache consistency between
simultaneous clients (basically, rules that determine what FUA and
flush from one client are able to guarantee for reads from another
client). When we don't permit simultaneous clients (such as qemu-nbd
without -e), the bit makes no sense; and for writable images, we
probably have a lot more work before we can declare that actions from
one client are cache-consistent with actions from another. But for
read-only images, where flush isn't changing any data, we might as
well advertise multi-conn support. What's more, advertisement of the
bit makes it easier for clients to determine if 'qemu-nbd -e' was in
use, where a second connection will succeed rather than hang until the
first client goes away.
This patch affects qemu as server in advertising the bit. We may want
to consider patches to qemu as client to attempt parallel connections
for higher throughput by spreading the load over those connections
when a server advertises multi-conn, but for now sticking to one
connection per nbd:// BDS is okay.
Peter Maydell [Thu, 5 Sep 2019 16:09:13 +0000 (17:09 +0100)]
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2019-09-05-v2' into staging
- Make the core libqtest library independent from global_qtest
- Clean up docs from hard-coded qemu-system-* names
- Install libattr-dev and libcap-dev in gitlab-ci to test virtio-9p
* remotes/huth-gitlab/tags/pull-request-2019-09-05-v2:
gitlab-ci.yml: Install libattr-devel and libcap-devel to test virtio-9p
qemu-doc: Do not hard-code the name of the QEMU binary
tests/vm: Take the J=x setting into account for the vm-boot-ssh targets, too
tests/libqtest: Use libqtest-single.h in tests that require global_qtest
tests/libqtest: Move global_test wrapper function into a separate header
tests: Remove unnecessary global_qtest references
tests/libqos: Replace clock_step with qtest_clock_step in virtio code
tests/libqos/e1000e: Make e1000e libqos functions independent from global_qtest
tests/migration: Do not use functions anymore that rely on global_qtest
Peter Maydell [Thu, 5 Sep 2019 15:33:39 +0000 (16:33 +0100)]
Merge remote-tracking branch 'remotes/berrange/tags/docs-pull-request' into staging
docs: add docs about use of automatic cleanup functions
This is ostensibly about adding docs for the g_autofree/g_autoptr
macros. As part of doing that, however, the existing HACKING doc
is merged into the CODING_STYLE doc and the text is converted to
rst with a table of contents.
# gpg: Signature made Thu 05 Sep 2019 14:43:44 BST
# gpg: using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <[email protected]>" [full]
# gpg: aka "Daniel P. Berrange <[email protected]>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF
* remotes/berrange/tags/docs-pull-request:
docs: split the CODING_STYLE doc into distinct groups
docs: document use of automatic cleanup functions in glib
docs: merge HACKING.rst contents into CODING_STYLE.rst
docs: convert README, CODING_STYLE and HACKING to RST syntax
Thomas Huth [Thu, 5 Sep 2019 10:36:50 +0000 (12:36 +0200)]
gitlab-ci.yml: Install libattr-devel and libcap-devel to test virtio-9p
So far the gitlab-ci was not testing virtio-9p yet, since we did not
install libattr-devel and libcap-devel in any of the pipelines. Do
it now to get some more test coverage.
There are only two remaining uses of gen_bx_im. In each case, we
know the destination mode -- not changing in the case of gen_jmp
or changing in the case of trans_BLX_i. Use this to simplify the
surrounding code.
For trans_BLX_i, use gen_jmp for the actual branch. For gen_jmp,
use gen_set_pc_im to set up the single-step.
We have been using store_reg and not store_reg_for_load when writing
back a loaded value into the base register. At first glance this is
incorrect when base == pc, however that case is UNPREDICTABLE.
Pass the T5 encoding of SUBS PC, LR, #IMM through the normal SUBS path
to make it clear exactly what's happening -- we hit ALUExceptionReturn
along that path.
The m-profile and a-profile decodings overlap. Only return false
for the case of wrong profile; handle UNDEFINED for permission failure
directly. This ensures that we don't accidentally pass an insn that
applies to the wrong profile.
projects / qemu.git / log