* remotes/thuth-gitlab/tags/pull-request-2021-06-02:
configure: bump min required CLang to 6.0 / XCode 10.0
configure: bump min required GCC to 7.5.0
configure: bump min required glib version to 2.56
tests/docker: drop CentOS 7 container
tests/vm: convert centos VM recipe to CentOS 8
crypto: drop used conditional check
crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
crypto: drop back compatibility typedefs for nettle
crypto: bump min nettle to 3.4, dropping RHEL-7 support
patchew: move quick build job from CentOS 7 to CentOS 8 container
block/ssh: Bump minimum libssh version to 0.8.7
docs: fix references to docs/devel/s390-dasd-ipl.rst
docs: fix references to docs/specs/tpm.rst
docs: fix references to docs/devel/build-system.rst
docs: fix references to docs/devel/atomics.rst
docs: fix references to docs/devel/tracing.rst
The glib version was not previously constrained by RHEL-7 since it
rebases fairly often. Instead SLES 12 and Ubuntu 16.04 were the
constraints in 00f2cfbbec63fb6f5a7789797a62ccedd22466ea. Both of
these are old enough that they are outside our platform support
matrix now.
crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gnutls version
crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gcrypt version and
assume that HMAC is always supported
crypto: bump min nettle to 3.4, dropping RHEL-7 support
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required nettle version and
drop a lot of backwards compatibility code for 2.x series of nettle.
Thomas Huth [Wed, 19 May 2021 15:58:59 +0000 (17:58 +0200)]
block/ssh: Bump minimum libssh version to 0.8.7
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. So from the RHEL-7 perspective, we do not have to support
libssh v0.7 anymore now.
Let's look at the versions from other distributions and operating
systems - according to repology.org, current shipping versions are:
* The version of libssh in Ubuntu 18.04 claims to be 0.8.0 from the
name of the package, but in reality it is a 0.7 patched up as a
Frankenstein monster with patches from the 0.8 development branch.
This gave us some headaches in the past already and so it never worked
with QEMU. All attempts to get it supported have failed in the past,
patches for QEMU have never been merged and a request to Ubuntu to
fix it in their 18.04 distro has been ignored:
Thus we really should ignore the libssh in Ubuntu 18.04 in QEMU, too.
Fix it by bumping the minimum libssh version to something that is
greater than 0.8.0 now. Debian Buster and openSUSE Leap have the
oldest version and so 0.8.7 is the new minimum.
Peter Maydell [Tue, 1 Jun 2021 20:23:26 +0000 (21:23 +0100)]
Merge remote-tracking branch 'remotes/ehabkost-gl/tags/x86-next-pull-request' into staging
x86 queue, 2021-06-01
Features:
* Add CPU model versions supporting 'xsaves' (Vitaly Kuznetsov)
* Support AVX512 ZMM regs dump (Robert Hoo)
Bug fixes:
* Use better matching family/model/stepping for generic CPUs
(Daniel P. Berrangé)
Cleanups:
* Hyper-V feature initialization cleanup (Vitaly Kuznetsov)
* SEV firmware error list touchups (Connor Kuehl)
* Constify CPUCaches and X86CPUDefinition (Philippe Mathieu-Daudé)
* Document when features can be added to kvm_default_props
(Eduardo Habkost)
* remotes/ehabkost-gl/tags/x86-next-pull-request: (24 commits)
sev: add missing firmware error conditions
sev: use explicit indices for mapping firmware error codes to strings
target/i386/sev: add support to query the attestation report
i386: use global kvm_state in hyperv_enabled() check
i386: prefer system KVM_GET_SUPPORTED_HV_CPUID ioctl over vCPU's one
i386: adjust the expected KVM_GET_SUPPORTED_HV_CPUID array size
i386: switch hyperv_expand_features() to using error_setg()
i386: move eVMCS enablement to hyperv_init_vcpu()
i386: split hyperv_handle_properties() into hyperv_expand_features()/hyperv_fill_cpuids()
i386: introduce hv_cpuid_cache
i386: drop FEAT_HYPERV feature leaves
i386: introduce hv_cpuid_get_host()
i386: introduce hyperv_feature_supported()
i386: stop using env->features[] for filling Hyper-V CPUIDs
i386: always fill Hyper-V CPUID feature leaves from X86CPU data
i386: invert hyperv_spinlock_attempts setting logic with hv_passthrough
i386: keep hyperv_vendor string up-to-date
i386: use better matching family/model/stepping for 'max' CPU
i386: use better matching family/model/stepping for 'qemu64' CPU
i386/cpu_dump: support AVX512 ZMM regs dump
...
John Snow [Thu, 27 May 2021 21:17:15 +0000 (17:17 -0400)]
gitlab: add python linters to CI
Add a Python container that has just enough juice for us to run the
Python code quality analysis tools. Base this container on Fedora,
because Fedora has very convenient packaging for testing multiple Python
versions.
We need python3, pip (for pulling packages), pipenv and virtualenv for
creating virtual environments, and tox for running tests. make is needed
for running 'make check-tox' and 'make venv-check' targets. Python3.10
is needed explicitly because the tox package only pulls in 3.6-3.9, but
we wish to test the forthcoming release of Python as well to help
predict any problems. Lastly, we need gcc to compile PyPI packages that
may not have a binary distribution available.
Add two tests:
check-python-pipenv uses pipenv to test a frozen, very explicit set of
packages against our minimum supported python version, Python 3.6. This
test is not allowed to fail. The dependencies this test uses do not
change unless python/Pipfile.lock is changed.
check-python-tox uses tox to install the latest versions of required
python dependencies against a wide array of Python versions from 3.6 to
3.9, even including the yet-to-be-released Python 3.10. This test is
allowed to fail with a warning.
John Snow [Thu, 27 May 2021 21:17:14 +0000 (17:17 -0400)]
python: add tox support
This is intended to be a manually run, non-CI script.
Use tox to test the linters against all python versions from 3.6 to
3.10. This will only work if you actually have those versions installed
locally, but Fedora makes this easy:
Unlike the pipenv tests (make venv-check), this pulls "whichever"
versions of the python packages, so they are unpinned and may break as
time goes on. In the case that breakages are found, setup.cfg should be
amended accordingly to avoid the bad dependant versions, or the code
should be amended to work around the issue.
With confidence that the tests pass on 3.6 through 3.10 inclusive, add
the appropriate classifiers to setup.cfg to indicate which versions we
claim to support.
Tox 3.18.0 or above is required to use the 'allowlist_externals' option.
John Snow [Thu, 27 May 2021 21:17:13 +0000 (17:17 -0400)]
python: add .gitignore
Ignore *Python* build and package output (build, dist, qemu.egg-info);
these files are not created as part of a QEMU build. They are created by
running the commands 'python3 setup.py <sdist|bdist>' when preparing
tarballs to upload to e.g. PyPI.
Ignore miscellaneous cached python confetti (mypy, pylint, et al)
Ignore .idea (pycharm) .vscode, and .venv (pipenv et al).
John Snow [Thu, 27 May 2021 21:17:12 +0000 (17:17 -0400)]
python: add Makefile for some common tasks
Add "make venv" to create the pipenv-managed virtual environment that
contains our explicitly pinned dependencies.
Add "make check" to run the python linters [in the host execution
environment].
Add "make venv-check" which combines the above two: create/update the
venv, then run the linters in that explicitly managed environment.
Add "make develop" which canonizes the runes needed to get both the
linting pre-requisites (the "[devel]" part), and the editable
live-install (the "-e" part) of these python libraries.
make clean: delete miscellaneous python packaging output possibly
created by pipenv, pip, or other python packaging utilities
make distclean: delete the above, the .venv, and the editable "qemu"
package forwarder (qemu.egg-info) if there is one.
John Snow [Thu, 27 May 2021 21:17:11 +0000 (17:17 -0400)]
python: add avocado-framework and tests
Try using avocado to manage our various tests; even though right now
they're only invoking shell scripts and not really running any
python-native code.
Create tests/, and add shell scripts which call out to mypy, flake8,
pylint and isort to enforce the standards in this directory.
Add avocado-framework to the setup.cfg development dependencies, and add
avocado.cfg to store some preferences for how we'd like the test output
to look.
Finally, add avocado-framework to the Pipfile environment and lock the
new dependencies. We are using avocado >= 87.0 here to take advantage of
some features that Cleber has helpfully added to make the test output
here *very* friendly and easy to read for developers that might chance
upon the output in Gitlab CI.
[Note: ALL of the dependencies get updated to the most modern versions
that exist at the time of this writing. No way around it that I have
seen. Not ideal, but so it goes.]
Provided you have the right development dependencies (mypy, flake8,
isort, pylint, and now avocado-framework) You should be able to run
"avocado --config avocado.cfg run tests/" from the python folder to run
all of these linters with the correct arguments.
(A forthcoming commit adds the much easier 'make check'.)
John Snow [Thu, 27 May 2021 21:17:10 +0000 (17:17 -0400)]
python: add devel package requirements to setuptools
setuptools doesn't have a formal understanding of development requires,
but it has an optional feataures section. Fine; add a "devel" feature
and add the requirements to it.
To avoid duplication, we can modify pipenv to install qemu[devel]
instead. This enables us to run invocations like "pip install -e
.[devel]" and test the package on bleeding-edge packages beyond those
specified in Pipfile.lock.
Importantly, this also allows us to install the qemu development
packages in a non-networked mode: `pip3 install --no-index -e .[devel]`
will now fail if the proper development dependencies are not already
met. This can be useful for automated build scripts where fetching
network packages may be undesirable.
John Snow [Thu, 27 May 2021 21:17:09 +0000 (17:17 -0400)]
python/qemu: add qemu package itself to pipenv
This adds the python qemu packages themselves to the pipenv manifest.
'pipenv sync' will create a virtual environment sufficient to use the SDK.
'pipenv sync --dev' will create a virtual environment sufficient to use
and test the SDK (with pylint, mypy, isort, flake8, etc.)
The qemu packages are installed in 'editable' mode; all changes made to
the python package inside the git tree will be reflected in the
installed package without reinstallation. This includes changes made
via git pull and so on.
John Snow [Thu, 27 May 2021 21:17:08 +0000 (17:17 -0400)]
python/qemu: add isort to pipenv
isort 5.0.0 through 5.0.4 has a bug that causes it to misinterpret
certain "from ..." clauses that are not related to imports.
isort < 5.1.1 has a bug where it does not handle comments near import
statements correctly.
Require 5.1.2 or greater.
isort can be run (in "check" mode) with 'isort -c qemu' from the python
root. isort can also be used to fix/rewrite import order automatically
by using 'isort qemu'.
John Snow [Thu, 27 May 2021 21:17:06 +0000 (17:17 -0400)]
python: add mypy to pipenv
0.730 appears to be about the oldest version that works with the
features we want, including nice human readable output (to make sure
iotest 297 passes), and type-parameterized Popen generics.
0.770, however, supports adding 'strict' to the config file, so require
at least 0.770.
Now that we are checking a namespace package, we need to tell mypy to
allow PEP420 namespaces, so modify the mypy config as part of the move.
mypy can now be run from the python root by typing 'mypy -p qemu'.
A note on mypy invocation: Running it as "mypy qemu/" changes the import
path detection mechanisms in mypy slightly, and it will fail. See
https://github.com/python/mypy/issues/8584 for a decent entry point with
more breadcrumbs on the various behaviors that contribute to this subtle
difference.
John Snow [Thu, 27 May 2021 21:17:04 +0000 (17:17 -0400)]
python: Add flake8 to pipenv
flake8 3.5.x does not support the --extend-ignore syntax used in the
.flake8 file to gracefully extend default ignores, so 3.6.x is our
minimum requirement. There is no known upper bound.
flake8 can be run from the python/ directory with no arguments.
John Snow [Thu, 27 May 2021 21:17:03 +0000 (17:17 -0400)]
python: add excluded dirs to flake8 config
Instruct flake8 to avoid certain well-known directories created by
python tooling that it ought not check.
Note that at-present, nothing actually creates a ".venv" directory; but
it is in such widespread usage as a de-facto location for a developer's
virtual environment that it should be excluded anyway. A forthcoming
commit canonizes this with a "make venv" command.
John Snow [Thu, 27 May 2021 21:17:02 +0000 (17:17 -0400)]
python: move flake8 config to setup.cfg
Update the comment concerning the flake8 exception to match commit 42c0dd12, whose commit message stated:
A note on the flake8 exception: flake8 will warn on *any* bare except,
but pylint's is context-aware and will suppress the warning if you
re-raise the exception.
John Snow [Thu, 27 May 2021 21:17:01 +0000 (17:17 -0400)]
python: add pylint to pipenv
We are specifying >= pylint 2.8.x for several reasons:
1. For setup.cfg support, added in pylint 2.5.x
2. To specify a version that has incompatibly dropped
bad-whitespace checks (2.6.x)
3. 2.7.x fixes "unsubscriptable" warnings in Python 3.9
4. 2.8.x adds a new, incompatible 'consider-using-with'
warning that must be disabled in some cases.
These pragmas cause warnings themselves in 2.7.x.
John Snow [Thu, 27 May 2021 21:16:58 +0000 (17:16 -0400)]
python: Add pipenv support
pipenv is a tool used for managing virtual environments with pinned,
explicit dependencies. It is used for precisely recreating python
virtual environments.
pipenv uses two files to do this:
(1) Pipfile, which is similar in purpose and scope to what setup.cfg
lists. It specifies the requisite minimum to get a functional
environment for using this package.
(2) Pipfile.lock, which is similar in purpose to `pip freeze >
requirements.txt`. It specifies a canonical virtual environment used for
deployment or testing. This ensures that all users have repeatable
results.
The primary benefit of using this tool is to ensure *rock solid*
repeatable CI results with a known set of packages. Although I endeavor
to support as many versions as I can, the fluid nature of the Python
toolchain often means tailoring code for fairly specific versions.
Note that pipenv is *not* required to install or use this module; this is
purely for the sake of repeatable testing by CI or developers.
Here, a "blank" pipfile is added with no dependencies, but specifies
Python 3.6 for the virtual environment.
Pipfile will specify our version minimums, while Pipfile.lock specifies
an exact loadout of packages that were known to operate correctly. This
latter file provides the real value for easy setup of container images
and CI environments.
John Snow [Thu, 27 May 2021 21:16:57 +0000 (17:16 -0400)]
python: add MANIFEST.in
When creating a source or binary distribution via 'python3 setup.py
<sdist|bdist>', the VERSION and PACKAGE.rst files aren't bundled by
default. Create a MANIFEST.in file that instructs the build tools to
include these so that installation from these files won't fail.
This is required by 'tox', as well as by the tooling needed to upload
packages to PyPI.
Exclude the 'README.rst' file -- that's intended as a guidebook to our
source tree, not a file that needs to be distributed.
John Snow [Thu, 27 May 2021 21:16:56 +0000 (17:16 -0400)]
python: add directory structure README.rst files
Add short readmes to python/, python/qemu/, python/qemu/machine,
python/qemu/qmp, and python/qemu/utils that explain the directory
hierarchy. These readmes are visible when browsing the source on
e.g. gitlab/github and are designed to help new developers/users quickly
make sense of the source tree.
They are not designed for inclusion in a published manual.
John Snow [Thu, 27 May 2021 21:16:55 +0000 (17:16 -0400)]
python: add VERSION file
Python infrastructure as it exists today is not capable reliably of
single-sourcing a package version from a parent directory. The authors
of pip are working to correct this, but as of today this is not possible.
The problem is that when using pip to build and install a python
package, it copies files over to a temporary directory and performs its
build there. This loses access to any information in the parent
directory, including git itself.
Further, Python versions have a standard (PEP 440) that may or may not
follow QEMU's versioning. In general, it does; but naturally QEMU does
not follow PEP 440. To avoid any automatically-generated conflict, a
manual version file is preferred.
I am proposing:
- Python tooling follows the QEMU version, indirectly, but with a major
version of 0 to indicate that the API is not expected to be
stable. This would mean version 0.5.2.0, 0.5.1.1, 0.5.3.0, etc.
- In the event that a Python package needs to be updated independently
of the QEMU version, a pre-release alpha version should be preferred,
but *only* after inclusion to the qemu development or stable branches.
e.g. 0.5.2.0a1, 0.5.2.0a2, and so on should be preferred prior to
5.2.0's release.
- The Python core tooling makes absolutely no version compatibility
checks or constraints. It *may* work with releases of QEMU from the
past or future, but it is not required to.
i.e., "qemu.machine" will, for now, remain in lock-step with QEMU.
- We reserve the right to split the qemu package into independently
versioned subpackages at a later date. This might allow for us to
begin versioning QMP independently from QEMU at a later date, if
we so choose.
Implement this versioning scheme by adding a VERSION file and setting it
to 0.6.0.0a1.
John Snow [Thu, 27 May 2021 21:16:54 +0000 (17:16 -0400)]
python: add qemu package installer
Add setup.cfg and setup.py, necessary for installing a package via
pip. Add a ReST document (PACKAGE.rst) explaining the basics of what
this package is for and who to contact for more information. This
document will be used as the landing page for the package on PyPI.
List the subpackages we intend to package by name instead of using
find_namespace because find_namespace will naively also packages tests,
things it finds in the dist/ folder, etc. I could not figure out how to
modify this behavior; adding allow/deny lists to setuptools kept
changing the packaged hierarchy. This works, roll with it.
I am not yet using a pyproject.toml style package manifest, because
"editable" installs are not defined (yet?) by PEP-517/518.
I consider editable installs crucial for development, though they have
(apparently) always been somewhat poorly defined.
Pip now (19.2 and later) now supports editable installs for projects
using pyproject.toml manifests, but might require the use of the
--no-use-pep517 flag, which somewhat defeats the point. Full support for
setup.py-less editable installs was not introduced until pip 21.1.1:
https://github.com/pypa/pip/pull/9547/commits/7a95720e796a5e56481c1cc20b6ce6249c50f357
For now, while the dust settles, stick with the de-facto
setup.py/setup.cfg combination supported by setuptools. It will be worth
re-evaluating this point again in the future when our supported build
platforms all ship a fairly modern pip.
John Snow [Thu, 27 May 2021 21:16:53 +0000 (17:16 -0400)]
python: create qemu packages
move python/qemu/*.py to python/qemu/[machine, qmp, utils]/*.py and
update import directives across the tree.
This is done to create a PEP420 namespace package, in which we may
create subpackages. To do this, the namespace directory ("qemu") should
not have any modules in it. Those files will go into new 'machine',
'qmp' and 'utils' subpackages instead.
Implement machine/__init__.py making the top-level classes and functions
from its various modules available directly inside the package. Change
qmp.py to qmp/__init__.py similarly, such that all of the useful QMP
library classes are available directly from "qemu.qmp" instead of
"qemu.qmp.qmp".
John Snow [Thu, 27 May 2021 21:16:52 +0000 (17:16 -0400)]
iotests/297: add --namespace-packages to mypy arguments
mypy is kind of weird about how it handles imports. For legacy reasons,
it won't load PEP 420 namespaces, because of logic implemented prior to
that becoming a standard.
So, if you plan on using any, you have to pass
--namespace-packages. Alright, fine.
John Snow [Thu, 27 May 2021 21:16:49 +0000 (17:16 -0400)]
python/machine: Disable pylint warning for open() in _pre_launch
Shift the open() call later so that the pylint pragma applies *only* to
that one open() call. Add a note that suggests why this is safe: the
resource is unconditionally cleaned up in _post_shutdown().
_post_shutdown is called after failed launches (see launch()), and
unconditionally after every call to shutdown(), and therefore also on
__exit__.
Willian Rampazzo [Thu, 20 May 2021 20:47:47 +0000 (17:47 -0300)]
acceptance tests: bump Avocado version to 88.1
Besides some internal changes, new features, and bug fixes, on the QEMU side,
this version fixes the following message seen when running the acceptance
tests: "Error running method "pre_tests" of plugin "fetchasset": 'bytes'
object has no attribute 'encode'".
The release notes are available at
https://avocado-framework.readthedocs.io/en/latest/releases/88_0.html.
Cleber Rosa [Mon, 12 Apr 2021 04:46:43 +0000 (00:46 -0400)]
Acceptance Tests: introduce CPU hotplug test
Even though there are qtest based tests for hotplugging CPUs (from
which this test took some inspiration from), this one adds checks
from a Linux guest point of view.
It should also serve as an example for tests that follow a similar
pattern and need to interact with QEMU (via qmp) and with the Linux
guest via SSH.
The LinuxTest class' launch_and_wait() method now behaves the same way
as this test's custom launch_vm(), so let's just use the upper layer
(common) method.
Cleber Rosa [Mon, 12 Apr 2021 04:46:40 +0000 (00:46 -0400)]
Acceptance Tests: set up SSH connection by default after boot for LinuxTest
The LinuxTest specifically targets users that need to interact with Linux
guests. So, it makes sense to give a connection by default, and avoid
requiring it as boiler-plate code.
Cleber Rosa [Mon, 12 Apr 2021 04:46:39 +0000 (00:46 -0400)]
Acceptance Tests: make username/password configurable
This makes the username/password used for authentication configurable,
because some guest operating systems may have restrictions on accounts
to be used for logins, and it just makes it better documented.
Cleber Rosa [Mon, 12 Apr 2021 04:46:37 +0000 (00:46 -0400)]
Acceptance Tests: move useful ssh methods to base class
Both the virtiofs submounts and the linux ssh mips malta tests
contains useful methods related to ssh that deserve to be made
available to other tests. Let's move them to an auxiliary, mix-in
class that will be used on the base LinuxTest class.
The method that helps with setting up an ssh connection will now
support both key and password based authentication, defaulting to key
based.
Cleber Rosa [Mon, 12 Apr 2021 04:46:36 +0000 (00:46 -0400)]
Python: add utility function for retrieving port redirection
Slightly different versions for the same utility code are currently
present on different locations. This unifies them all, giving
preference to the version from virtiofs_submounts.py, because of the
last tweaks added to it.
While at it, this adds a "qemu.utils" module to host the utility
function and a test.
Cleber Rosa [Mon, 12 Apr 2021 04:46:35 +0000 (00:46 -0400)]
tests/acceptance/virtiofs_submounts.py: evaluate string not length
If the vmlinuz variable is set to anything that evaluates to True,
then the respective arguments should be set. If the variable contains
an empty string, than it will evaluate to False, and the extra
arguments will not be set.
This keeps the same logic, but improves readability a bit.
Cleber Rosa [Thu, 11 Feb 2021 22:01:42 +0000 (17:01 -0500)]
Python: expose QEMUMachine's temporary directory
Each instance of qemu.machine.QEMUMachine currently has a "test
directory", which may not have any relation to a "test", and it's
really a temporary directory.
Users instantiating the QEMUMachine class will be able to set the
location of the directory that will *contain* the QEMUMachine unique
temporary directory, so that parameter name has been changed from
test_dir to base_temp_dir.
A property has been added to allow users to access it without using
private attributes, and with that, the directory is created on first
use of the property.
The SEV userspace header[1] exports a couple of other error conditions that
aren't listed in QEMU's SEV implementation, so let's just round out the
list.
target/i386/sev: add support to query the attestation report
The SEV FW >= 0.23 added a new command that can be used to query the
attestation report containing the SHA-256 digest of the guest memory
and VMSA encrypted with the LAUNCH_UPDATE and sign it with the PEK.
Note, we already have a command (LAUNCH_MEASURE) that can be used to
query the SHA-256 digest of the guest memory encrypted through the
LAUNCH_UPDATE. The main difference between previous and this command
is that the report is signed with the PEK and unlike the LAUNCH_MEASURE
command the ATTESATION_REPORT command can be called while the guest
is running.
Add a QMP interface "query-sev-attestation-report" that can be used
to get the report encoded in base64.
i386: use global kvm_state in hyperv_enabled() check
There is no need to use vCPU-specific kvm state in hyperv_enabled() check
and we need to do that when feature expansion happens early, before vCPU
specific KVM state is created.
i386: prefer system KVM_GET_SUPPORTED_HV_CPUID ioctl over vCPU's one
KVM_GET_SUPPORTED_HV_CPUID was made a system wide ioctl which can be called
prior to creating vCPUs and we are going to use that to expand Hyper-V cpu
features early. Use it when it is supported by KVM.
i386: adjust the expected KVM_GET_SUPPORTED_HV_CPUID array size
SYNDBG leaves were recently (Linux-5.8) added to KVM but we haven't
updated the expected size of KVM_GET_SUPPORTED_HV_CPUID output in
KVM so we now make serveral tries before succeeding. Update the
default.
i386: split hyperv_handle_properties() into hyperv_expand_features()/hyperv_fill_cpuids()
The intention is to call hyperv_expand_features() early, before vCPUs
are created and use the acquired data later when we set guest visible
CPUID data.
Just like with cpuid_cache, it makes no sense to call
KVM_GET_SUPPORTED_HV_CPUID more than once and instead of (ab)using
env->features[] and/or trying to keep all the code in one place, it is
better to introduce persistent hv_cpuid_cache and hv_cpuid_get_host()
accessor to it.
Note, hv_cpuid_get_fw() is converted to using hv_cpuid_get_host()
just to be removed later with Hyper-V specific feature words.
Hyper-V feature leaves are weird. We have some of them in
feature_word_info[] array but we don't use feature_word_info
magic to enable them. Neither do we use feature_dependencies[]
mechanism to validate the configuration as it doesn't allign
well with Hyper-V's many-to-many dependency chains. Some of
the feature leaves hold not only feature bits, but also values.
E.g. FEAT_HV_NESTED_EAX contains both features and the supported
Enlightened VMCS range.
Hyper-V features are already represented in 'struct X86CPU' with
uint64_t hyperv_features so duplicating them in env->features adds
little (or zero) benefits. THe other half of Hyper-V emulation features
is also stored with values in hyperv_vendor_id[], hyperv_limits[],...
so env->features[] is already incomplete.
Remove Hyper-V feature leaves from env->features[] completely.
kvm_hyperv_properties[] is converted to using raw CPUID func/reg
pairs for features, this allows us to get rid of hv_cpuid_get_fw()
conversion.
i386: stop using env->features[] for filling Hyper-V CPUIDs
As a preparatory patch to dropping Hyper-V CPUID leaves from
feature_word_info[] stop using env->features[] as a temporary
storage of Hyper-V CPUIDs, just build Hyper-V CPUID leaves directly
from kvm_hyperv_properties[] data.
i386: always fill Hyper-V CPUID feature leaves from X86CPU data
We have all the required data in X86CPU already and as we are about to
split hyperv_handle_properties() into hyperv_expand_features()/
hyperv_fill_cpuids() we can remove the blind copy. The functional change
is that QEMU won't pass CPUID leaves it doesn't currently know about
to the guest but arguably this is a good change.
When cpu->hyperv_vendor is not set manually we default to "Microsoft Hv"
and in 'hv_passthrough' mode we get the information from the host. This
information is stored in cpu->hyperv_vendor_id[] array but we don't update
cpu->hyperv_vendor string so e.g. QMP's query-cpu-model-expansion output
is incorrect.
i386: use better matching family/model/stepping for 'max' CPU
The 'max' CPU under TCG currently reports a family/model/stepping that
approximately corresponds to an AMD K7 vintage architecture.
The K7 series predates the introduction of 64-bit support by AMD
in the K8 series. This has been reported to lead to LLVM complaints
about generating 64-bit code for a 32-bit CPU target
LLVM ERROR: 64-bit code requested on a subtarget that doesn't support it!
It appears LLVM looks at the family/model/stepping, despite qemu64
reporting it is 64-bit capable.
This patch changes 'max' to report a CPUID with the family, model
and stepping taken from a
i386: use better matching family/model/stepping for 'qemu64' CPU
The 'qemu64' CPUID currently reports a family/model/stepping that
approximately corresponds to an AMD K7 vintage architecture.
The K7 series predates the introduction of 64-bit support by AMD
in the K8 series. This has been reported to lead to LLVM complaints
about generating 64-bit code for a 32-bit CPU target
LLVM ERROR: 64-bit code requested on a subtarget that doesn't support it!
It appears LLVM looks at the family/model/stepping, despite qemu64
reporting it is 64-bit capable.
This patch changes 'qemu64' to report a CPUID with the family, model
and stepping taken from a
Robert Hoo [Wed, 21 Apr 2021 06:23:52 +0000 (14:23 +0800)]
i386/cpu_dump: support AVX512 ZMM regs dump
Since commit fa4518741e (target-i386: Rename struct XMMReg to ZMMReg),
CPUX86State.xmm_regs[] has already been extended to 512bit to support
AVX512.
Also, other qemu level supports for AVX512 registers are there for
years.
But in x86_cpu_dump_state(), still only dump XMM registers no matter
YMM/ZMM is enabled.
This patch is to complement this, let it dump XMM/YMM/ZMM accordingly.
i386: Document when features can be added to kvm_default_props
It's very easy to mistakenly extend kvm_default_props to include
features that require a kernel version that's too recent. Add a
comment warning about that, pointing to the documentation file
where the minimum kernel version for KVM is documented.
target/i386: Add CPU model versions supporting 'xsaves'
Hyper-V 2016 refuses to boot on Skylake+ CPU models because they lack
'xsaves'/'vmx-xsaves' features and this diverges from real hardware. The
same issue emerges with AMD "EPYC" CPU model prior to version 3 which got
'xsaves' added. EPYC-Rome/EPYC-Milan CPU models have 'xsaves' enabled from
the very beginning so the comment blaming KVM to explain why other CPUs
lack 'xsaves' is likely outdated.
* remotes/kraxel/tags/usb-20210528-pull-request:
hw/usb: hcd-xhci-pci: Fix spec violation of IP flag for MSI/MSI-X
hw/usb: hcd-xhci-pci: Raise MSI/MSI-X interrupts only when told to
hw/input/hid: Add support for keys of jp106 keyboard.
Peter Maydell [Fri, 28 May 2021 21:55:56 +0000 (22:55 +0100)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
# gpg: Signature made Thu 27 May 2021 04:06:17 BST
# gpg: using RSA key EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <[email protected]>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* remotes/jasowang/tags/net-pull-request:
tap-bsd: Remove special casing for older OpenBSD releases
virtio-net: failover: add missing remove_migration_state_change_notifier()
hw/net/imx_fec: return 0xffff when accessing non-existing PHY
Peter Maydell [Fri, 28 May 2021 15:25:20 +0000 (16:25 +0100)]
Merge remote-tracking branch 'remotes/rth-gitlab/tags/pull-tcg-20210526' into staging
Adjust types for some memory access functions.
Reduce inclusion of tcg headers.
Fix watchpoints vs replay.
Fix tcg/aarch64 roli expansion.
Introduce SysemuCPUOps structure.
# gpg: Signature made Thu 27 May 2021 00:43:54 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "[email protected]"
# gpg: Good signature from "Richard Henderson <[email protected]>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F
* remotes/rth-gitlab/tags/pull-tcg-20210526: (31 commits)
hw/core: Constify TCGCPUOps
target/mips: Fold jazz behaviour into mips_cpu_do_transaction_failed
cpu: Move CPUClass::get_paging_enabled to SysemuCPUOps
cpu: Move CPUClass::get_memory_mapping to SysemuCPUOps
cpu: Move CPUClass::get_phys_page_debug to SysemuCPUOps
cpu: Move CPUClass::asidx_from_attrs to SysemuCPUOps
cpu: Move CPUClass::write_elf* to SysemuCPUOps
cpu: Move CPUClass::get_crash_info to SysemuCPUOps
cpu: Move CPUClass::virtio_is_big_endian to SysemuCPUOps
cpu: Move CPUClass::vmsd to SysemuCPUOps
cpu: Introduce SysemuCPUOps structure
cpu: Move AVR target vmsd field from CPUClass to DeviceClass
cpu: Rename CPUClass vmsd -> legacy_vmsd
cpu: Assert DeviceClass::vmsd is NULL on user emulation
cpu: Directly use get_memory_mapping() fallback handlers in place
cpu: Directly use get_paging_enabled() fallback handlers in place
cpu: Directly use cpu_write_elf*() fallback handlers in place
cpu: Introduce cpu_virtio_is_big_endian()
cpu: Un-inline cpu_get_phys_page_debug and cpu_asidx_from_attrs
cpu: Split as cpu-common / cpu-sysemu
...
Ruimei Yan [Fri, 21 May 2021 02:42:24 +0000 (10:42 +0800)]
hw/usb: hcd-xhci-pci: Fix spec violation of IP flag for MSI/MSI-X
Per xHCI spec v1.2 chapter 4.17.5 page 296:
If MSI or MSI-X interrupts are enabled, Interrupt Pending (IP)
shall be cleared automatically when the PCI dword write generated
by the interrupt assertion is complete.
Currently QEMU does not clear the IP flag in the MSI / MSI-X mode.
This causes subsequent spurious interrupt to be delivered to guests.
To solve this, we change the xhci intr_raise() hook routine to have
a bool return value that is passed to its caller (the xhci core),
with true indicating that IP should be self-cleared.
Ruimei Yan [Fri, 21 May 2021 02:42:23 +0000 (10:42 +0800)]
hw/usb: hcd-xhci-pci: Raise MSI/MSI-X interrupts only when told to
At present MSI / MSI-X interrupts are triggered regardless of the
irq level. We should have checked the level to determine whether
the interrupt needs to be delivered.
The level check logic was present in early versions of the xhci
model, but got dropped later by a rework of interrupt handling
under commit 4c4abe7cc903 ("xhci: rework interrupt handling").
hw/input/hid: Add support for keys of jp106 keyboard.
Add support for the following keys: KATAKANAHIRAGANA, HENKAN, MUHENKAN,
RO, and YEN. Before this commit, these keys did not work as expected
when a jp106 keyboard was connected to the guest as a usb-kbd device.
Peter Maydell [Thu, 27 May 2021 15:32:57 +0000 (16:32 +0100)]
Merge remote-tracking branch 'remotes/philmd/tags/gitlab-ci-20210527' into staging
GitLab CI patches queue
- Explode .gitlab-ci.yml in reusable templates
- Add job to cross build/test TCI on i386 host
- Remove CentOS 7 linux-user build job
- Temporarily set Avocado-based jobs in manual mode
- Increase time to hold Avocado reports to 1 week
# gpg: Signature made Thu 27 May 2021 15:40:58 BST
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <[email protected]>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE
* remotes/philmd/tags/gitlab-ci-20210527:
gitlab: Convert check-dco/check-patch jobs to the 'rules' syntax
gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master'
gitlab: Run Avocado tests manually (except mainstream CI)
gitlab: Keep Avocado reports during 1 week
gitlab: Extract cross-container jobs to container-cross.yml
gitlab: Document how forks can use different set of jobs
gitlab: Move current job set to qemu-project.yml
gitlab: Extract all default build/test jobs to buildtest.yml
gitlab: Drop linux user build job for CentOS 7
gitlab: Extract core container jobs to container-core.yml
gitlab: Extract default build/test jobs templates
gitlab: Extract build stages to stages.yml
gitlab: Extract DCO/style check jobs to static_checks.yml
gitlab: Extract crossbuild job templates to crossbuild-template.yml
gitlab: Extract container job template to container-template.yml
gitlab: Enable cross-i386 builds of TCI
gitlab: Rename ACCEL_CONFIGURE_OPTS to EXTRA_CONFIGURE_OPTS
gitlab: Replace YAML anchors by extends (acceptance_test_job)
gitlab: Use $CI_DEFAULT_BRANCH instead of hardcoded 'master'
We want to skip the checkpatch and DCO signoff jobs when
pushing to the default branch. Currently this branch is
called 'master', but we don't need to hardcode this in
the CI configuration, because the $CI_DEFAULT_BRANCH
env variable exposes it.
gitlab: Run Avocado tests manually (except mainstream CI)
Due to a design problem and misunderstanding between the Avocado
framework and QEMU, Avocado is fetching many asset artifacts it
shouldn't be fetching, exhausting the jobs CI timeout.
Since Avocado artifacts are cached, this is not an issue with old
forks, which already have populated the cache and do not need to
download new artifacts to run the tests.
However this is very confusing to new contributors who start to
fork the project and keep having failing CI pipelines.
As a temporary kludge, add the QEMU_CI_AVOCADO_TESTING variable
to allow old forks to keep running the Avocado tests, while still
allowing new forks to use the mainstream set of CI tests.
Keep the tests enabled by default on the mainstream namespace
which is old enough to have a populated cache, hoping we will
keep this cache long enough until the Avocado/QEMU design issue
is fixed.
projects / qemu.git / log