]> Git Repo - buildroot-mgba.git/blame - docs/manual/contribute.txt
support/scripts/mkusers: allow option for system uid/gid
[buildroot-mgba.git] / docs / manual / contribute.txt
CommitLineData
a79a54a0 1// -*- mode:doc; -*-
3edb0271 2// vim: set syntax=asciidoc:
a79a54a0 3
569db405 4== Contributing to Buildroot
37b22be9
TDS
5
6There are many ways in which you can contribute to Buildroot: analyzing
7and fixing bugs, analyzing and fixing package build failures detected by
8the autobuilders, testing and reviewing patches sent by other
9developers, working on the items in our TODO list and sending your own
10improvements to Buildroot or its manual. The following sections give a
11little more detail on each of these items.
12
13If you are interested in contributing to Buildroot, the first thing you
14should do is to subscribe to the Buildroot mailing list. This list is
15the main way of interacting with other Buildroot developers and to send
16contributions to. If you aren't subscribed yet, then refer to
713134fc 17xref:community-resources[] for the subscription link.
a79a54a0 18
37b22be9
TDS
19If you are going to touch the code, it is highly recommended to use a
20git repository of Buildroot, rather than starting from an extracted
21source code tarball. Git is the easiest way to develop from and directly
22send your patches to the mailing list. Refer to xref:getting-buildroot[]
23for more information on obtaining a Buildroot git tree.
a79a54a0 24
569db405 25=== Reproducing, analyzing and fixing bugs
e640872e
TDS
26
27A first way of contributing is to have a look at the open bug reports in
162b30bc 28the https://bugs.buildroot.org/buglist.cgi?product=buildroot[Buildroot bug
e640872e
TDS
29tracker]. As we strive to keep the bug count as small as possible, all
30help in reproducing, analyzing and fixing reported bugs is more than
31welcome. Don't hesitate to add a comment to bug reports reporting your
32findings, even if you don't yet see the full picture.
33
569db405 34=== Analyzing and fixing autobuild failures
f7d5d971
TDS
35
36The Buildroot autobuilders are a set of build machines that continuously
37run Buildroot builds based on random configurations. This is done for
38all architectures supported by Buildroot, with various toolchains, and
39with a random selection of packages. With the large commit activity on
40Buildroot, these autobuilders are a great help in detecting problems
41very early after commit.
42
43All build results are available at http://autobuild.buildroot.org[],
44statistics are at http://autobuild.buildroot.org/stats.php[]. Every day,
45an overview of all failed packages is sent to the mailing list.
46
47Detecting problems is great, but obviously these problems have to be
48fixed as well. Your contribution is very welcome here! There are
49basically two things that can be done:
50
51- Analyzing the problems. The daily summary mails do not contain details
52 about the actual failures: in order to see what's going on you have to
53 open the build log and check the last output. Having someone doing
54 this for all packages in the mail is very useful for other developers,
55 as they can make a quick initial analysis based on this output alone.
56
57- Fixing a problem. When fixing autobuild failures, you should follow
58 these steps:
59 . Check if you can reproduce the problem by building with the same
60 configuration. You can do this manually, or use the
61 http://git.buildroot.org/buildroot-test/tree/utils/br-reproduce-build[br-reproduce-build]
62 script that will automatically clone a Buildroot git repository,
63 checkout the correct revision, download and set the right
64 configuration, and start the build.
65 . Analyze the problem and create a fix.
66 . Verify that the problem is really fixed by starting from a clean
67 Buildroot tree and only applying your fix.
68 . Send the fix to the Buildroot mailing list (see
69 xref:submitting-patches[]). In case you created a patch against the
70 package sources, you should also send the patch upstream so that the
71 problem will be fixed in a later release, and the patch in Buildroot
72 can be removed.
73 In the commit message of a patch fixing an autobuild failure, add a
74 reference to the build result directory, as follows:
75---------------------
e875c2ff 76Fixes: http://autobuild.buildroot.org/results/51000a9d4656afe9e0ea6f07b9f8ed374c2e4069
f7d5d971
TDS
77---------------------
78
569db405 79=== Reviewing and testing patches
aeae2356
TDS
80
81With the amount of patches sent to the mailing list each day, the
82maintainer has a very hard job to judge which patches are ready to apply
83and which ones aren't. Contributors can greatly help here by reviewing
84and testing these patches.
85
86In the review process, do not hesitate to respond to patch submissions
87for remarks, suggestions or anything that will help everyone to
88understand the patches and make them better. Please use internet
89style replies in plain text emails when responding to patch
90submissions.
91
92To indicate approval of a patch, there are three formal tags that keep
93track of this approval. To add your tag to a patch, reply to it with the
94approval tag below the original author's Signed-off-by line. These tags
95will be picked up automatically by patchwork (see
f9ce8d17 96xref:apply-patches-patchwork[]) and will be part of the commit log when
aeae2356
TDS
97the patch is accepted.
98
99Tested-by:: Indicates that the patch has been tested successfully.
100 You are encouraged to specify what kind of testing you performed
101 (compile-test on architecture X and Y, runtime test on target A,
102 ...). This additional information helps other testers and the
103 maintainer.
104
105Reviewed-by:: Indicates that you code-reviewed the patch and did your
106 best in spotting problems, but you are not sufficiently familiar with
107 the area touched to provide an Acked-by tag. This means that there
108 may be remaining problems in the patch that would be spotted by
109 someone with more experience in that area. Should such problems be
110 detected, your Reviewed-by tag remains appropriate and you cannot
111 be blamed.
112
113Acked-by:: Indicates that you code-reviewed the patch and you are
114 familiar enough with the area touched to feel that the patch can be
115 committed as-is (no additional changes required). In case it later
116 turns out that something is wrong with the patch, your Acked-by could
117 be considered inappropriate. The difference between Acked-by and
118 Reviewed-by is thus mainly that you are prepared to take the blame on
119 Acked patches, but not on Reviewed ones.
120
121If you reviewed a patch and have comments on it, you should simply reply
122to the patch stating these comments, without providing a Reviewed-by or
123Acked-by tag. These tags should only be provided if you judge the patch
124to be good as it is.
125
126It is important to note that neither Reviewed-by nor Acked-by imply
127that testing has been performed. To indicate that you both reviewed and
128tested the patch, provide two separate tags (Reviewed/Acked-by and
129Tested-by).
130
131Note also that _any developer_ can provide Tested/Reviewed/Acked-by
132tags, without exception, and we encourage everyone to do this. Buildroot
133does not have a defined group of _core_ developers, it just so happens
134that some developers are more active than others. The maintainer will
135value tags according to the track record of their submitter. Tags
136provided by a regular contributor will naturally be trusted more than
137tags provided by a newcomer. As you provide tags more regularly, your
138'trustworthiness' (in the eyes of the maintainer) will go up, but _any_
139tag provided is valuable.
140
141Buildroot's Patchwork website can be used to pull in patches for testing
142purposes. Please see xref:apply-patches-patchwork[] for more
143information on using Buildroot's Patchwork website to apply patches.
144
713134fc
TDS
145[[apply-patches-patchwork]]
146==== Applying Patches from Patchwork
147
148The main use of Buildroot's Patchwork website for a developer is for
149pulling in patches into their local git repository for testing
150purposes.
151
152When browsing patches in the patchwork management interface, an +mbox+
153link is provided at the top of the page. Copy this link address and
154run the following commands:
155
156---------------------
157$ git checkout -b <test-branch-name>
158$ wget -O - <mbox-url> | git am
159---------------------
160
161Another option for applying patches is to create a bundle. A bundle is
162a set of patches that you can group together using the patchwork
163interface. Once the bundle is created and the bundle is made public,
164you can copy the +mbox+ link for the bundle and apply the bundle
165using the above commands.
166
167
569db405 168=== Work on items from the TODO list
b77143a7
TDS
169
170If you want to contribute to Buildroot but don't know where to start,
171and you don't like any of the above topics, you can always work on items
172from the http://elinux.org/Buildroot#Todo_list[Buildroot TODO list].
173Don't hesitate to discuss an item first on the mailing list or on IRC.
174Do edit the wiki to indicate when you start working on an item, so we
175avoid duplicate efforts.
aeae2356 176
a79a54a0 177[[submitting-patches]]
569db405 178=== Submitting patches
a79a54a0 179
37b22be9
TDS
180[NOTE]
181_Please, do not attach patches to bugs, send them to the mailing list
182instead_.
183
1133097c 184If you made some changes to Buildroot and you would like to contribute
4311eb00
AV
185them to the Buildroot project, proceed as follows.
186
eb56faf2
AV
187==== The formatting of a patch
188
189We expect patches to be formatted in a specific way. This is necessary
190to make it easy to review patches, to be able to apply them easily to
191the git repository, to make it easy to find back in the history how
192and why things have changed, and to make it possible to use +git
193bisect+ to locate the origin of a problem.
194
195First of all, it is essential that the patch has a good commit
196message. The commit message should start with a separate line with a
74fc5dce
TP
197brief summary of the change, prefixed by the area touched by the
198patch. A few examples of good commit titles:
199
200* +package/linuxptp: bump version to 2.0+
201
202* +configs/imx23evk: bump Linux version to 4.19+
203
204* +package/pkg-generic: postpone evaluation of dependency conditions+
205
206* +boot/uboot: needs host-{flex,bison}+
207
208* +support/testing: add python-ubjson tests+
209
210The description that follows the prefix should start with a lower case
211letter (i.e "bump", "needs", "postpone", "add" in the above examples).
212
213Second, the body of the commit message should describe _why_ this
eb56faf2
AV
214change is needed, and if necessary also give details about _how_ it
215was done. When writing the commit message, think of how the reviewers
216will read it, but also think about how you will read it when you look
217at this change again a few years down the line.
218
74fc5dce 219Third, the patch itself should do only one change, but do it
eb56faf2
AV
220completely. Two unrelated or weakly related changes should usually be
221done in two separate patches. This usually means that a patch affects
222only a single package. If several changes are related, it is often
223still possible to split them up in small patches and apply them in a
224specific order. Small patches make it easier to review, and often
225make it easier to understand afterwards why a change was done.
226However, each patch must be complete. It is not allowed that the
227build is broken when only the first but not the second patch is
228applied. This is necessary to be able to use +git bisect+ afterwards.
229
230Of course, while you're doing your development, you're probably going
231back and forth between packages, and certainly not committing things
232immediately in a way that is clean enough for submission. So most
233developers rewrite the history of commits to produce a clean set of
234commits that is appropriate for submission. To do this, you need to
235use _interactive rebasing_. You can learn about it
236https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History[in the Pro
237Git book]. Sometimes, it is even easier to discard you history with
238+git reset --soft origin/master+ and select individual changes with
239+git add -i+ or +git add -p+.
240
241Finally, the patch should be signed off. This is done by adding
242+Signed-off-by: Your Real Name <[email protected]>+ at the end of the
243commit message. +git commit -s+ does that for you, if configured
244properly. The +Signed-off-by+ tag means that you publish the patch
bedb3be1 245under the Buildroot license (i.e. GPL-2.0+, except for package patches,
eb56faf2
AV
246which have the upstream license), and that you are allowed to do so.
247See http://developercertificate.org/[the Developer Certificate of
248Origin] for details.
249
250When adding new packages, you should submit every package in a
251separate patch. This patch should have the update to
252+package/Config.in+, the package +Config.in+ file, the +.mk+ file, the
253+.hash+ file, any init script, and all package patches. If the package
254has many sub-options, these are sometimes better added as separate
255follow-up patches. The summary line should be something like
256+<packagename>: new package+. The body of the commit message can be
257empty for simple packages, or it can contain the description of the
258package (like the Config.in help text). If anything special has to be
259done to build the package, this should also be explained explicitly in
260the commit message body.
261
262When you bump a package to a new version, you should also submit a
263separate patch for each package. Don't forget to update the +.hash+
264file, or add it if it doesn't exist yet. Also don't forget to check if
265the +_LICENSE+ and +_LICENSE_FILES+ are still valid. The summary line
266should be something like +<packagename>: bump to version <new
267version>+. If the new version only contains security updates compared
268to the existing one, the summary should be +<packagename>: security
269bump to version <new version>+ and the commit message body should show
270the CVE numbers that are fixed. If some package patches can be removed
271in the new version, it should be explained explicitly why they can be
272removed, preferably with the upstream commit ID. Also any other
273required changes should be explained explicitly, like configure
274options that no longer exist or are no longer needed.
275
589cbc61
TP
276If you are interested in getting notified of build failures and of
277further changes in the packages you added or modified, please add
8209d722
JK
278yourself to the DEVELOPERS file. This should be done in the same patch
279creating or modifying the package. See xref:DEVELOPERS[the DEVELOPERS file]
280for more information.
589cbc61 281
01e9dd29
RM
282Buildroot provides a handy tool to check for common coding style
283mistakes on files you created or modified, called +check-package+ (see
284xref:check-package[] for more information).
285
4311eb00
AV
286==== Preparing a patch series
287
288Starting from the changes committed in your local git view, _rebase_
289your development branch on top of the upstream tree before generating
290a patch set. To do so, run:
a79a54a0
SM
291
292---------------------
5664c4af
SM
293$ git fetch --all --tags
294$ git rebase origin/master
a79a54a0
SM
295---------------------
296
1133097c 297Now, you are ready to generate then submit your patch set.
a79a54a0
SM
298
299To generate it, run:
300
301---------------------
5664c4af 302$ git format-patch -M -n -s -o outgoing origin/master
a79a54a0
SM
303---------------------
304
305This will generate patch files in the +outgoing+ subdirectory,
f7da9ee5 306automatically adding the +Signed-off-by+ line.
a79a54a0 307
a79a54a0 308Once patch files are generated, you can review/edit the commit message
1133097c 309before submitting them, using your favorite text editor.
a79a54a0 310
6017cb8f 311Buildroot provides a handy tool to know to whom your patches should be
589cbc61
TP
312sent, called +get-developers+ (see xref:DEVELOPERS[] for more
313information). This tool reads your patches and outputs the appropriate
314+git send-email+ command to use:
a79a54a0
SM
315
316---------------------
7ca9fc31 317$ ./utils/get-developers outgoing/*
6017cb8f
TP
318---------------------
319
320Use the output of +get-developers+ to send your patches:
321
322---------------------
323$ git send-email --to [email protected] --cc bob --cc alice outgoing/*
a79a54a0
SM
324---------------------
325
507d56a6
PK
326Alternatively, +get-developers -e+ can be used directly with the
327+--cc-cmd+ argument to +git send-email+ to automatically CC the
328affected developers:
329
330---------------------
331$ git send-email --to [email protected] \
332 --cc-cmd './utils/get-developers -e' origin/master
333---------------------
334
335+git+ can be configured to automatically do this out of the box with:
336
337---------------------
338$ git config sendemail.to [email protected]
339$ git config sendemail.ccCmd "$(pwd)/utils/get-developers -e"
340---------------------
341
342And then just do:
343
344---------------------
345$ git send-email origin/master
346---------------------
347
a79a54a0
SM
348Note that +git+ should be configured to use your mail account.
349To configure +git+, see +man git-send-email+ or google it.
350
1133097c
TDS
351If you do not use +git send-email+, make sure posted *patches are not
352line-wrapped*, otherwise they cannot easily be applied. In such a case,
353fix your e-mail client, or better yet, learn to use +git send-email+.
a79a54a0 354
569db405 355==== Cover letter
9cbeb341
VT
356
357If you want to present the whole patch set in a separate mail, add
358+--cover-letter+ to the +git format-patch+ command (see +man
359git-format-patch+ for further information). This will generate a
360template for an introduction e-mail to your patch series.
361
362A 'cover letter' may be useful to introduce the changes you propose
363in the following cases:
364
365* large number of commits in the series;
366
367* deep impact of the changes in the rest of the project;
368
369* RFC footnote:[RFC: (Request for comments) change proposal];
370
371* whenever you feel it will help presenting your work, your choices,
372 the review process, etc.
373
76ed6949
BM
374==== Patches for maintenance branches
375
376When fixing bugs on a maintenance branch, bugs should be fixed on the
377master branch first. The commit log for such a patch may then contain a
378post-commit note specifying what branches are affected:
379
380----
381package/foo: fix stuff
382
383Signed-off-by: Your Real Name <[email protected]>
384---
385Backport to: 2020.02.x, 2020.05.x
386(2020.08.x not affected as the version was bumped)
387----
388
389Those changes will then be backported by a maintainer to the affected
390branches.
391
392However, some bugs may apply only to a specific release, for example
393because it is using an older version of a package. In that case, patches
394should be based off the maintenance branch, and the patch subject prefix
395must include the maintenance branch name (for example "[PATCH 2020.02.x]").
396This can be done with the +git format-patch+ flag +--subject-prefix+:
397
398---------------------
399$ git format-patch --subject-prefix "PATCH 2020.02.x" \
400 -M -s -o outgoing origin/2020.02.x
401---------------------
402
403Then send the patches with +git send-email+, as described above.
404
569db405 405==== Patch revision changelog
9cbeb341
VT
406
407When improvements are requested, the new revision of each commit
408should include a changelog of the modifications between each
409submission. Note that when your patch series is introduced by a cover
9b556772
TDS
410letter, an overall changelog may be added to the cover letter in
411addition to the changelog in the individual commits.
b41ec531
WB
412The best thing to rework a patch series is by interactive rebasing:
413+git rebase -i origin/master+. Consult the git manual for more
414information.
9cbeb341
VT
415
416When added to the individual commits, this changelog is added when
3b1df656 417editing the commit message. Below the +Signed-off-by+ section, add
9cbeb341
VT
418+---+ and your changelog.
419
420Although the changelog will be visible for the reviewers in the mail
421thread, as well as in http://patchwork.buildroot.org[patchwork], +git+
422will automatically ignores lines below +---+ when the patch will be
423merged. This is the intended behavior: the changelog is not meant to
424be preserved forever in the +git+ history of the project.
425
426Hereafter the recommended layout:
427
428---------------
4319fbef 429Patch title: short explanation, max 72 chars
9cbeb341 430
4319fbef
TDS
431A paragraph that explains the problem, and how it manifests itself. If
432the problem is complex, it is OK to add more paragraphs. All paragraphs
433should be wrapped at 72 characters.
9cbeb341 434
4319fbef 435A paragraph that explains the root cause of the problem. Again, more
0c48904f 436than one paragraph is OK.
4319fbef
TDS
437
438Finally, one or more paragraphs that explain how the problem is solved.
439Don't hesitate to explain complex solutions in detail.
440
441Signed-off-by: John DOE <[email protected]>
9cbeb341 442
9cbeb341
VT
443---
444Changes v2 -> v3:
445 - foo bar (suggested by Jane)
446 - bar buz
447
448Changes v1 -> v2:
449 - alpha bravo (suggested by John)
450 - charly delta
451---------------
452
453Any patch revision should include the version number. The version number
454is simply composed of the letter +v+ followed by an +integer+ greater or
455equal to two (i.e. "PATCH v2", "PATCH v3" ...).
456
457This can be easily handled with +git format-patch+ by using the option
458+--subject-prefix+:
459
460---------------------
461$ git format-patch --subject-prefix "PATCH v4" \
ffc9e468 462 -M -s -o outgoing origin/master
9cbeb341
VT
463---------------------
464
59fa7b7f
FN
465Since git version 1.8.1, you can also use +-v <n>+ (where <n> is the
466version number):
467
468---------------------
469$ git format-patch -v4 -M -s -o outgoing origin/master
470---------------------
471
76bcbae3
VOR
472When you provide a new version of a patch, please mark the old one as
473superseded in http://patchwork.buildroot.org[patchwork]. You need to
474create an account on http://patchwork.buildroot.org[patchwork] to be
475able to modify the status of your patches. Note that you can only change
476the status of patches you submitted yourself, which means the email
477address you register in http://patchwork.buildroot.org[patchwork] should
478match the one you use for sending patches to the mailing list.
479
59fa7b7f
FN
480You can also add the +--in-reply-to <message-id>+ option when
481submitting a patch to the mailing list. The id of the mail to reply to
482can be found under the "Message Id" tag on
483http://patchwork.buildroot.org[patchwork]. The advantage of
484*in-reply-to* is that patchwork will automatically mark the previous
485version of the patch as superseded.
486
a79a54a0 487[[reporting-bugs]]
569db405 488=== Reporting issues/bugs or getting help
a79a54a0 489
713134fc
TDS
490Before reporting any issue, please check in
491xref:community-resources[the mailing list archive] whether someone has
492already reported and/or fixed a similar problem.
a79a54a0 493
1c97e35f 494However you choose to report bugs or get help, either by
713134fc
TDS
495opening a bug in the xref:community-resources[bug tracker] or by
496xref:community-resources[sending a mail to the mailing list], there are
ecd23535 497a number of details to provide in order to help people reproduce and
a79a54a0
SM
498find a solution to the issue.
499
ecd23535 500Try to think as if you were trying to help someone else; in
a79a54a0
SM
501that case, what would you need?
502
503Here is a short list of details to provide in such case:
504
505* host machine (OS/release)
506* version of Buildroot
507* target for which the build fails
1c97e35f 508* package(s) for which the build fails
a79a54a0
SM
509* the command that fails and its output
510* any information you think that may be relevant
511
1c97e35f 512Additionally, you should add the +.config+ file (or if you know how, a
eddeec8c 513+defconfig+; see xref:customize-store-buildroot-config[]).
a79a54a0 514
ecd23535 515If some of these details are too large, do not hesitate to use a
1c97e35f
TDS
516pastebin service. Note that not all available pastebin services will
517preserve Unix-style line terminators when downloading raw pastes.
518Following pastebin services are known to work correctly:
519- https://gist.github.com/
520- http://code.bulix.org/
e2e57d56 521
a9dc2de5 522=== Using the runtime tests framework
e2e57d56 523
a9dc2de5
TP
524Buildroot includes a run-time testing framework built upon Python
525scripting and QEMU runtime execution. The goals of the framework are
e2e57d56
MW
526the following:
527
a9dc2de5 528* build a well defined Buildroot configuration
e2e57d56 529* optionally, verify some properties of the build output
a9dc2de5
TP
530* optionally, boot the build results under Qemu, and verify that a
531 given feature is working as expected
e2e57d56 532
a9dc2de5
TP
533The entry point to use the runtime tests framework is the
534+support/testing/run-tests+ tool, which has a series of options
535documented in the tool's help '-h' description. Some common options
536include setting the download folder, the output folder, keeping build
537output, and for multiple test cases, you can set the JLEVEL for each.
e2e57d56
MW
538
539Here is an example walk through of running a test case.
540
541* For a first step, let us see what all the test case options are. The test
542cases can be listed by executing +support/testing/run-tests -l+. These tests
543can all be run individually during test development from the console. Both
544one at a time and selectively as a group of a subset of tests.
545
546---------------------
547$ support/testing/run-tests -l
548List of tests
549test_run (tests.utils.test_check_package.TestCheckPackage)
e2e57d56
MW
550test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootMusl) ... ok
551test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc) ... ok
552test_run (tests.toolchain.test_external.TestExternalToolchainCCache) ... ok
553test_run (tests.toolchain.test_external.TestExternalToolchainCtngMusl) ... ok
554test_run (tests.toolchain.test_external.TestExternalToolchainLinaroArm) ... ok
555test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv4) ... ok
556test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv5) ... ok
557test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv7) ... ok
558[snip]
559test_run (tests.init.test_systemd.TestInitSystemSystemdRoFull) ... ok
560test_run (tests.init.test_systemd.TestInitSystemSystemdRoIfupdown) ... ok
561test_run (tests.init.test_systemd.TestInitSystemSystemdRoNetworkd) ... ok
562test_run (tests.init.test_systemd.TestInitSystemSystemdRwFull) ... ok
563test_run (tests.init.test_systemd.TestInitSystemSystemdRwIfupdown) ... ok
564test_run (tests.init.test_systemd.TestInitSystemSystemdRwNetworkd) ... ok
565test_run (tests.init.test_busybox.TestInitSystemBusyboxRo) ... ok
566test_run (tests.init.test_busybox.TestInitSystemBusyboxRoNet) ... ok
567test_run (tests.init.test_busybox.TestInitSystemBusyboxRw) ... ok
568test_run (tests.init.test_busybox.TestInitSystemBusyboxRwNet) ... ok
569
570Ran 157 tests in 0.021s
571
572OK
573---------------------
574
a9dc2de5 575* Then, to run one test case:
e2e57d56
MW
576
577---------------------
578$ support/testing/run-tests -d dl -o output_folder -k tests.init.test_busybox.TestInitSystemBusyboxRw
57915:03:26 TestInitSystemBusyboxRw Starting
58015:03:28 TestInitSystemBusyboxRw Building
58115:08:18 TestInitSystemBusyboxRw Building done
58215:08:27 TestInitSystemBusyboxRw Cleaning up
583.
584Ran 1 test in 301.140s
585
586OK
587---------------------
588
a9dc2de5
TP
589The standard output indicates if the test is successful or not. By
590default, the output folder for the test is deleted automatically
591unless the option +-k+ is passed to *keep* the output directory.
e2e57d56 592
a9dc2de5
TP
593==== Creating a test case
594
595Within the Buildroot repository, the testing framework is organized at the
596top level in +support/testing/+ by folders of +conf+, +infra+ and +tests+.
597All the test cases live under the +tests+ folder and are organized in various
598folders representing the category of test.
599
600The best way to get familiar with how to create a test case is to look
601at a few of the basic file system +support/testing/tests/fs/+ and init
602+support/testing/tests/init/+ test scripts. Those tests give good
603examples of a basic tests that include both checking the build
604results, and doing runtime tests. There are other more advanced cases
605that use things like nested +br2-external+ folders to provide
606skeletons and additional packages.
607
608Creating a basic test case involves:
609
610* Defining a test class that inherits from +infra.basetest.BRTest+
611
612* Defining the +config+ member of the test class, to the Buildroot
613 configuration to build for this test case. It can optionally rely on
614 configuration snippets provided by the runtime test infrastructure:
615 +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ to get a basic
616 architecture/toolchain configuration, and
617 +infra.basetest.MINIMAL_CONFIG+ to not build any filesystem. The
618 advantage of using +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ is that a
619 matching Linux kernel image is provided, which allows to boot the
620 resulting image in Qemu without having to build a Linux kernel image
621 as part of the test case, therefore significant decreasing the build
622 time required for the test case.
623
624* Implementing a +def test_run(self):+ function to implement the
625 actual tests to run after the build has completed. They may be tests
626 that verify the build output, by running command on the host using
627 the +run_cmd_on_host()+ helper function. Or they may boot the
628 generated system in Qemu using the +Emulator+ object available as
629 +self.emulator+ in the test case. For example +self.emulator.boot()+
630 allows to boot the system in Qemu, +self.emulator.login()+ allows to
631 login, +self.emulator.run()+ allows to run shell commands inside
632 Qemu.
633
634After creating the test script, add yourself to the +DEVELOPERS+ file to
635be the maintainer of that test case.
636
637==== Debugging a test case
638
639When a test case runs, the +output_folder+ will contain the following:
e2e57d56
MW
640
641---------------------
642$ ls output_folder/
643TestInitSystemBusyboxRw/
644TestInitSystemBusyboxRw-build.log
645TestInitSystemBusyboxRw-run.log
646---------------------
647
a9dc2de5
TP
648+TestInitSystemBusyboxRw/+ is the Buildroot output directory, and it
649is preserved only if the +-k+ option is passed.
650
651+TestInitSystemBusyboxRw-build.log+ is the log of the Buildroot build.
652
653+TestInitSystemBusyboxRw-run.log+ is the log of the Qemu boot and
654test. This file will only exist if the build was successful and the
655test case involves booting under Qemu.
656
657If you want to manually run Qemu to do manual tests of the build
658result, the first few lines of +TestInitSystemBusyboxRw-run.log+
659contain the Qemu command line to use.
660
661You can also make modifications to the current sources inside the
662+output_folder+ (e.g. for debug purposes) and rerun the standard
663Buildroot make targets (in order to regenerate the complete image with
664the new modifications) and then rerun the test.
665
666==== Runtime tests and Gitlab CI
667
668All runtime tests are regularly executed by Buildroot Gitlab CI
669infrastructure, see .gitlab.yml and
670https://gitlab.com/buildroot.org/buildroot/-/jobs.
671
672You can also use Gitlab CI to test your new test cases, or verify that
673existing tests continue to work after making changes in Buildroot.
e2e57d56 674
a9dc2de5
TP
675In order to achieve this, you need to create a fork of the Buildroot
676project on Gitlab, and be able to push branches to your Buildroot fork
677on Gitlab.
e2e57d56 678
a9dc2de5
TP
679The name of the branch that you push will determine if a Gitlab CI
680pipeline will be triggered or not, and for which test cases.
e2e57d56 681
a9dc2de5
TP
682In the examples below, the <name> component of the branch name is an
683arbitrary string you choose.
e2e57d56 684
a9dc2de5
TP
685* To trigger all run-test test case jobs, push a branch that ends with
686 +-runtime-tests+:
e2e57d56
MW
687
688---------------------
689 $ git push gitlab HEAD:<name>-runtime-tests
690---------------------
691
a9dc2de5
TP
692* To trigger one or several test case jobs, push a branch that ends
693 with the complete test case name
694 (+tests.init.test_busybox.TestInitSystemBusyboxRo+) or with the name
695 of a category of tests (+tests.init.test_busybox+):
e2e57d56
MW
696
697---------------------
698 $ git push gitlab HEAD:<name>-<test case name>
699---------------------
23186356
TP
700
701Example to run one test:
702
703---------------------
704 $ git push gitlab HEAD:foo-tests.init.test_busybox.TestInitSystemBusyboxRo
705---------------------
706
707Examples to run several tests part of the same group:
708
709---------------------
710 $ git push gitlab HEAD:foo-tests.init.test_busybox
711 $ git push gitlab HEAD:foo-tests.init
712---------------------
This page took 0.871362 seconds and 4 git commands to generate.