]>
Commit | Line | Data |
---|---|---|
a79a54a0 | 1 | // -*- mode:doc; -*- |
3edb0271 | 2 | // vim: set syntax=asciidoc: |
a79a54a0 | 3 | |
569db405 | 4 | == Contributing to Buildroot |
37b22be9 TDS |
5 | |
6 | There are many ways in which you can contribute to Buildroot: analyzing | |
7 | and fixing bugs, analyzing and fixing package build failures detected by | |
8 | the autobuilders, testing and reviewing patches sent by other | |
9 | developers, working on the items in our TODO list and sending your own | |
10 | improvements to Buildroot or its manual. The following sections give a | |
11 | little more detail on each of these items. | |
12 | ||
13 | If you are interested in contributing to Buildroot, the first thing you | |
14 | should do is to subscribe to the Buildroot mailing list. This list is | |
15 | the main way of interacting with other Buildroot developers and to send | |
16 | contributions to. If you aren't subscribed yet, then refer to | |
713134fc | 17 | xref:community-resources[] for the subscription link. |
a79a54a0 | 18 | |
37b22be9 TDS |
19 | If you are going to touch the code, it is highly recommended to use a |
20 | git repository of Buildroot, rather than starting from an extracted | |
21 | source code tarball. Git is the easiest way to develop from and directly | |
22 | send your patches to the mailing list. Refer to xref:getting-buildroot[] | |
23 | for more information on obtaining a Buildroot git tree. | |
a79a54a0 | 24 | |
569db405 | 25 | === Reproducing, analyzing and fixing bugs |
e640872e TDS |
26 | |
27 | A first way of contributing is to have a look at the open bug reports in | |
162b30bc | 28 | the https://bugs.buildroot.org/buglist.cgi?product=buildroot[Buildroot bug |
e640872e TDS |
29 | tracker]. As we strive to keep the bug count as small as possible, all |
30 | help in reproducing, analyzing and fixing reported bugs is more than | |
31 | welcome. Don't hesitate to add a comment to bug reports reporting your | |
32 | findings, even if you don't yet see the full picture. | |
33 | ||
569db405 | 34 | === Analyzing and fixing autobuild failures |
f7d5d971 TDS |
35 | |
36 | The Buildroot autobuilders are a set of build machines that continuously | |
37 | run Buildroot builds based on random configurations. This is done for | |
38 | all architectures supported by Buildroot, with various toolchains, and | |
39 | with a random selection of packages. With the large commit activity on | |
40 | Buildroot, these autobuilders are a great help in detecting problems | |
41 | very early after commit. | |
42 | ||
43 | All build results are available at http://autobuild.buildroot.org[], | |
44 | statistics are at http://autobuild.buildroot.org/stats.php[]. Every day, | |
45 | an overview of all failed packages is sent to the mailing list. | |
46 | ||
47 | Detecting problems is great, but obviously these problems have to be | |
48 | fixed as well. Your contribution is very welcome here! There are | |
49 | basically two things that can be done: | |
50 | ||
51 | - Analyzing the problems. The daily summary mails do not contain details | |
52 | about the actual failures: in order to see what's going on you have to | |
53 | open the build log and check the last output. Having someone doing | |
54 | this for all packages in the mail is very useful for other developers, | |
55 | as they can make a quick initial analysis based on this output alone. | |
56 | ||
57 | - Fixing a problem. When fixing autobuild failures, you should follow | |
58 | these steps: | |
59 | . Check if you can reproduce the problem by building with the same | |
60 | configuration. You can do this manually, or use the | |
61 | http://git.buildroot.org/buildroot-test/tree/utils/br-reproduce-build[br-reproduce-build] | |
62 | script that will automatically clone a Buildroot git repository, | |
63 | checkout the correct revision, download and set the right | |
64 | configuration, and start the build. | |
65 | . Analyze the problem and create a fix. | |
66 | . Verify that the problem is really fixed by starting from a clean | |
67 | Buildroot tree and only applying your fix. | |
68 | . Send the fix to the Buildroot mailing list (see | |
69 | xref:submitting-patches[]). In case you created a patch against the | |
70 | package sources, you should also send the patch upstream so that the | |
71 | problem will be fixed in a later release, and the patch in Buildroot | |
72 | can be removed. | |
73 | In the commit message of a patch fixing an autobuild failure, add a | |
74 | reference to the build result directory, as follows: | |
75 | --------------------- | |
e875c2ff | 76 | Fixes: http://autobuild.buildroot.org/results/51000a9d4656afe9e0ea6f07b9f8ed374c2e4069 |
f7d5d971 TDS |
77 | --------------------- |
78 | ||
569db405 | 79 | === Reviewing and testing patches |
aeae2356 TDS |
80 | |
81 | With the amount of patches sent to the mailing list each day, the | |
82 | maintainer has a very hard job to judge which patches are ready to apply | |
83 | and which ones aren't. Contributors can greatly help here by reviewing | |
84 | and testing these patches. | |
85 | ||
86 | In the review process, do not hesitate to respond to patch submissions | |
87 | for remarks, suggestions or anything that will help everyone to | |
88 | understand the patches and make them better. Please use internet | |
89 | style replies in plain text emails when responding to patch | |
90 | submissions. | |
91 | ||
92 | To indicate approval of a patch, there are three formal tags that keep | |
93 | track of this approval. To add your tag to a patch, reply to it with the | |
94 | approval tag below the original author's Signed-off-by line. These tags | |
95 | will be picked up automatically by patchwork (see | |
f9ce8d17 | 96 | xref:apply-patches-patchwork[]) and will be part of the commit log when |
aeae2356 TDS |
97 | the patch is accepted. |
98 | ||
99 | Tested-by:: Indicates that the patch has been tested successfully. | |
100 | You are encouraged to specify what kind of testing you performed | |
101 | (compile-test on architecture X and Y, runtime test on target A, | |
102 | ...). This additional information helps other testers and the | |
103 | maintainer. | |
104 | ||
105 | Reviewed-by:: Indicates that you code-reviewed the patch and did your | |
106 | best in spotting problems, but you are not sufficiently familiar with | |
107 | the area touched to provide an Acked-by tag. This means that there | |
108 | may be remaining problems in the patch that would be spotted by | |
109 | someone with more experience in that area. Should such problems be | |
110 | detected, your Reviewed-by tag remains appropriate and you cannot | |
111 | be blamed. | |
112 | ||
113 | Acked-by:: Indicates that you code-reviewed the patch and you are | |
114 | familiar enough with the area touched to feel that the patch can be | |
115 | committed as-is (no additional changes required). In case it later | |
116 | turns out that something is wrong with the patch, your Acked-by could | |
117 | be considered inappropriate. The difference between Acked-by and | |
118 | Reviewed-by is thus mainly that you are prepared to take the blame on | |
119 | Acked patches, but not on Reviewed ones. | |
120 | ||
121 | If you reviewed a patch and have comments on it, you should simply reply | |
122 | to the patch stating these comments, without providing a Reviewed-by or | |
123 | Acked-by tag. These tags should only be provided if you judge the patch | |
124 | to be good as it is. | |
125 | ||
126 | It is important to note that neither Reviewed-by nor Acked-by imply | |
127 | that testing has been performed. To indicate that you both reviewed and | |
128 | tested the patch, provide two separate tags (Reviewed/Acked-by and | |
129 | Tested-by). | |
130 | ||
131 | Note also that _any developer_ can provide Tested/Reviewed/Acked-by | |
132 | tags, without exception, and we encourage everyone to do this. Buildroot | |
133 | does not have a defined group of _core_ developers, it just so happens | |
134 | that some developers are more active than others. The maintainer will | |
135 | value tags according to the track record of their submitter. Tags | |
136 | provided by a regular contributor will naturally be trusted more than | |
137 | tags provided by a newcomer. As you provide tags more regularly, your | |
138 | 'trustworthiness' (in the eyes of the maintainer) will go up, but _any_ | |
139 | tag provided is valuable. | |
140 | ||
141 | Buildroot's Patchwork website can be used to pull in patches for testing | |
142 | purposes. Please see xref:apply-patches-patchwork[] for more | |
143 | information on using Buildroot's Patchwork website to apply patches. | |
144 | ||
713134fc TDS |
145 | [[apply-patches-patchwork]] |
146 | ==== Applying Patches from Patchwork | |
147 | ||
148 | The main use of Buildroot's Patchwork website for a developer is for | |
149 | pulling in patches into their local git repository for testing | |
150 | purposes. | |
151 | ||
152 | When browsing patches in the patchwork management interface, an +mbox+ | |
153 | link is provided at the top of the page. Copy this link address and | |
154 | run the following commands: | |
155 | ||
156 | --------------------- | |
157 | $ git checkout -b <test-branch-name> | |
158 | $ wget -O - <mbox-url> | git am | |
159 | --------------------- | |
160 | ||
161 | Another option for applying patches is to create a bundle. A bundle is | |
162 | a set of patches that you can group together using the patchwork | |
163 | interface. Once the bundle is created and the bundle is made public, | |
164 | you can copy the +mbox+ link for the bundle and apply the bundle | |
165 | using the above commands. | |
166 | ||
167 | ||
569db405 | 168 | === Work on items from the TODO list |
b77143a7 TDS |
169 | |
170 | If you want to contribute to Buildroot but don't know where to start, | |
171 | and you don't like any of the above topics, you can always work on items | |
172 | from the http://elinux.org/Buildroot#Todo_list[Buildroot TODO list]. | |
173 | Don't hesitate to discuss an item first on the mailing list or on IRC. | |
174 | Do edit the wiki to indicate when you start working on an item, so we | |
175 | avoid duplicate efforts. | |
aeae2356 | 176 | |
a79a54a0 | 177 | [[submitting-patches]] |
569db405 | 178 | === Submitting patches |
a79a54a0 | 179 | |
37b22be9 TDS |
180 | [NOTE] |
181 | _Please, do not attach patches to bugs, send them to the mailing list | |
182 | instead_. | |
183 | ||
1133097c | 184 | If you made some changes to Buildroot and you would like to contribute |
4311eb00 AV |
185 | them to the Buildroot project, proceed as follows. |
186 | ||
eb56faf2 AV |
187 | ==== The formatting of a patch |
188 | ||
189 | We expect patches to be formatted in a specific way. This is necessary | |
190 | to make it easy to review patches, to be able to apply them easily to | |
191 | the git repository, to make it easy to find back in the history how | |
192 | and why things have changed, and to make it possible to use +git | |
193 | bisect+ to locate the origin of a problem. | |
194 | ||
195 | First of all, it is essential that the patch has a good commit | |
196 | message. The commit message should start with a separate line with a | |
74fc5dce TP |
197 | brief summary of the change, prefixed by the area touched by the |
198 | patch. A few examples of good commit titles: | |
199 | ||
200 | * +package/linuxptp: bump version to 2.0+ | |
201 | ||
202 | * +configs/imx23evk: bump Linux version to 4.19+ | |
203 | ||
204 | * +package/pkg-generic: postpone evaluation of dependency conditions+ | |
205 | ||
206 | * +boot/uboot: needs host-{flex,bison}+ | |
207 | ||
208 | * +support/testing: add python-ubjson tests+ | |
209 | ||
210 | The description that follows the prefix should start with a lower case | |
211 | letter (i.e "bump", "needs", "postpone", "add" in the above examples). | |
212 | ||
213 | Second, the body of the commit message should describe _why_ this | |
eb56faf2 AV |
214 | change is needed, and if necessary also give details about _how_ it |
215 | was done. When writing the commit message, think of how the reviewers | |
216 | will read it, but also think about how you will read it when you look | |
217 | at this change again a few years down the line. | |
218 | ||
74fc5dce | 219 | Third, the patch itself should do only one change, but do it |
eb56faf2 AV |
220 | completely. Two unrelated or weakly related changes should usually be |
221 | done in two separate patches. This usually means that a patch affects | |
222 | only a single package. If several changes are related, it is often | |
223 | still possible to split them up in small patches and apply them in a | |
224 | specific order. Small patches make it easier to review, and often | |
225 | make it easier to understand afterwards why a change was done. | |
226 | However, each patch must be complete. It is not allowed that the | |
227 | build is broken when only the first but not the second patch is | |
228 | applied. This is necessary to be able to use +git bisect+ afterwards. | |
229 | ||
230 | Of course, while you're doing your development, you're probably going | |
231 | back and forth between packages, and certainly not committing things | |
232 | immediately in a way that is clean enough for submission. So most | |
233 | developers rewrite the history of commits to produce a clean set of | |
234 | commits that is appropriate for submission. To do this, you need to | |
235 | use _interactive rebasing_. You can learn about it | |
236 | https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History[in the Pro | |
237 | Git book]. Sometimes, it is even easier to discard you history with | |
238 | +git reset --soft origin/master+ and select individual changes with | |
239 | +git add -i+ or +git add -p+. | |
240 | ||
241 | Finally, the patch should be signed off. This is done by adding | |
242 | +Signed-off-by: Your Real Name <[email protected]>+ at the end of the | |
243 | commit message. +git commit -s+ does that for you, if configured | |
244 | properly. The +Signed-off-by+ tag means that you publish the patch | |
bedb3be1 | 245 | under the Buildroot license (i.e. GPL-2.0+, except for package patches, |
eb56faf2 AV |
246 | which have the upstream license), and that you are allowed to do so. |
247 | See http://developercertificate.org/[the Developer Certificate of | |
248 | Origin] for details. | |
249 | ||
250 | When adding new packages, you should submit every package in a | |
251 | separate patch. This patch should have the update to | |
252 | +package/Config.in+, the package +Config.in+ file, the +.mk+ file, the | |
253 | +.hash+ file, any init script, and all package patches. If the package | |
254 | has many sub-options, these are sometimes better added as separate | |
255 | follow-up patches. The summary line should be something like | |
256 | +<packagename>: new package+. The body of the commit message can be | |
257 | empty for simple packages, or it can contain the description of the | |
258 | package (like the Config.in help text). If anything special has to be | |
259 | done to build the package, this should also be explained explicitly in | |
260 | the commit message body. | |
261 | ||
262 | When you bump a package to a new version, you should also submit a | |
263 | separate patch for each package. Don't forget to update the +.hash+ | |
264 | file, or add it if it doesn't exist yet. Also don't forget to check if | |
265 | the +_LICENSE+ and +_LICENSE_FILES+ are still valid. The summary line | |
266 | should be something like +<packagename>: bump to version <new | |
267 | version>+. If the new version only contains security updates compared | |
268 | to the existing one, the summary should be +<packagename>: security | |
269 | bump to version <new version>+ and the commit message body should show | |
270 | the CVE numbers that are fixed. If some package patches can be removed | |
271 | in the new version, it should be explained explicitly why they can be | |
272 | removed, preferably with the upstream commit ID. Also any other | |
273 | required changes should be explained explicitly, like configure | |
274 | options that no longer exist or are no longer needed. | |
275 | ||
589cbc61 TP |
276 | If you are interested in getting notified of build failures and of |
277 | further changes in the packages you added or modified, please add | |
8209d722 JK |
278 | yourself to the DEVELOPERS file. This should be done in the same patch |
279 | creating or modifying the package. See xref:DEVELOPERS[the DEVELOPERS file] | |
280 | for more information. | |
589cbc61 | 281 | |
01e9dd29 RM |
282 | Buildroot provides a handy tool to check for common coding style |
283 | mistakes on files you created or modified, called +check-package+ (see | |
284 | xref:check-package[] for more information). | |
285 | ||
4311eb00 AV |
286 | ==== Preparing a patch series |
287 | ||
288 | Starting from the changes committed in your local git view, _rebase_ | |
289 | your development branch on top of the upstream tree before generating | |
290 | a patch set. To do so, run: | |
a79a54a0 SM |
291 | |
292 | --------------------- | |
5664c4af SM |
293 | $ git fetch --all --tags |
294 | $ git rebase origin/master | |
a79a54a0 SM |
295 | --------------------- |
296 | ||
1133097c | 297 | Now, you are ready to generate then submit your patch set. |
a79a54a0 SM |
298 | |
299 | To generate it, run: | |
300 | ||
301 | --------------------- | |
5664c4af | 302 | $ git format-patch -M -n -s -o outgoing origin/master |
a79a54a0 SM |
303 | --------------------- |
304 | ||
305 | This will generate patch files in the +outgoing+ subdirectory, | |
f7da9ee5 | 306 | automatically adding the +Signed-off-by+ line. |
a79a54a0 | 307 | |
a79a54a0 | 308 | Once patch files are generated, you can review/edit the commit message |
1133097c | 309 | before submitting them, using your favorite text editor. |
a79a54a0 | 310 | |
6017cb8f | 311 | Buildroot provides a handy tool to know to whom your patches should be |
589cbc61 TP |
312 | sent, called +get-developers+ (see xref:DEVELOPERS[] for more |
313 | information). This tool reads your patches and outputs the appropriate | |
314 | +git send-email+ command to use: | |
a79a54a0 SM |
315 | |
316 | --------------------- | |
7ca9fc31 | 317 | $ ./utils/get-developers outgoing/* |
6017cb8f TP |
318 | --------------------- |
319 | ||
320 | Use the output of +get-developers+ to send your patches: | |
321 | ||
322 | --------------------- | |
323 | $ git send-email --to [email protected] --cc bob --cc alice outgoing/* | |
a79a54a0 SM |
324 | --------------------- |
325 | ||
507d56a6 PK |
326 | Alternatively, +get-developers -e+ can be used directly with the |
327 | +--cc-cmd+ argument to +git send-email+ to automatically CC the | |
328 | affected developers: | |
329 | ||
330 | --------------------- | |
331 | $ git send-email --to [email protected] \ | |
332 | --cc-cmd './utils/get-developers -e' origin/master | |
333 | --------------------- | |
334 | ||
335 | +git+ can be configured to automatically do this out of the box with: | |
336 | ||
337 | --------------------- | |
338 | $ git config sendemail.to [email protected] | |
339 | $ git config sendemail.ccCmd "$(pwd)/utils/get-developers -e" | |
340 | --------------------- | |
341 | ||
342 | And then just do: | |
343 | ||
344 | --------------------- | |
345 | $ git send-email origin/master | |
346 | --------------------- | |
347 | ||
a79a54a0 SM |
348 | Note that +git+ should be configured to use your mail account. |
349 | To configure +git+, see +man git-send-email+ or google it. | |
350 | ||
1133097c TDS |
351 | If you do not use +git send-email+, make sure posted *patches are not |
352 | line-wrapped*, otherwise they cannot easily be applied. In such a case, | |
353 | fix your e-mail client, or better yet, learn to use +git send-email+. | |
a79a54a0 | 354 | |
569db405 | 355 | ==== Cover letter |
9cbeb341 VT |
356 | |
357 | If you want to present the whole patch set in a separate mail, add | |
358 | +--cover-letter+ to the +git format-patch+ command (see +man | |
359 | git-format-patch+ for further information). This will generate a | |
360 | template for an introduction e-mail to your patch series. | |
361 | ||
362 | A 'cover letter' may be useful to introduce the changes you propose | |
363 | in the following cases: | |
364 | ||
365 | * large number of commits in the series; | |
366 | ||
367 | * deep impact of the changes in the rest of the project; | |
368 | ||
369 | * RFC footnote:[RFC: (Request for comments) change proposal]; | |
370 | ||
371 | * whenever you feel it will help presenting your work, your choices, | |
372 | the review process, etc. | |
373 | ||
76ed6949 BM |
374 | ==== Patches for maintenance branches |
375 | ||
376 | When fixing bugs on a maintenance branch, bugs should be fixed on the | |
377 | master branch first. The commit log for such a patch may then contain a | |
378 | post-commit note specifying what branches are affected: | |
379 | ||
380 | ---- | |
381 | package/foo: fix stuff | |
382 | ||
383 | Signed-off-by: Your Real Name <[email protected]> | |
384 | --- | |
385 | Backport to: 2020.02.x, 2020.05.x | |
386 | (2020.08.x not affected as the version was bumped) | |
387 | ---- | |
388 | ||
389 | Those changes will then be backported by a maintainer to the affected | |
390 | branches. | |
391 | ||
392 | However, some bugs may apply only to a specific release, for example | |
393 | because it is using an older version of a package. In that case, patches | |
394 | should be based off the maintenance branch, and the patch subject prefix | |
395 | must include the maintenance branch name (for example "[PATCH 2020.02.x]"). | |
396 | This can be done with the +git format-patch+ flag +--subject-prefix+: | |
397 | ||
398 | --------------------- | |
399 | $ git format-patch --subject-prefix "PATCH 2020.02.x" \ | |
400 | -M -s -o outgoing origin/2020.02.x | |
401 | --------------------- | |
402 | ||
403 | Then send the patches with +git send-email+, as described above. | |
404 | ||
569db405 | 405 | ==== Patch revision changelog |
9cbeb341 VT |
406 | |
407 | When improvements are requested, the new revision of each commit | |
408 | should include a changelog of the modifications between each | |
409 | submission. Note that when your patch series is introduced by a cover | |
9b556772 TDS |
410 | letter, an overall changelog may be added to the cover letter in |
411 | addition to the changelog in the individual commits. | |
b41ec531 WB |
412 | The best thing to rework a patch series is by interactive rebasing: |
413 | +git rebase -i origin/master+. Consult the git manual for more | |
414 | information. | |
9cbeb341 VT |
415 | |
416 | When added to the individual commits, this changelog is added when | |
3b1df656 | 417 | editing the commit message. Below the +Signed-off-by+ section, add |
9cbeb341 VT |
418 | +---+ and your changelog. |
419 | ||
420 | Although the changelog will be visible for the reviewers in the mail | |
421 | thread, as well as in http://patchwork.buildroot.org[patchwork], +git+ | |
422 | will automatically ignores lines below +---+ when the patch will be | |
423 | merged. This is the intended behavior: the changelog is not meant to | |
424 | be preserved forever in the +git+ history of the project. | |
425 | ||
426 | Hereafter the recommended layout: | |
427 | ||
428 | --------------- | |
4319fbef | 429 | Patch title: short explanation, max 72 chars |
9cbeb341 | 430 | |
4319fbef TDS |
431 | A paragraph that explains the problem, and how it manifests itself. If |
432 | the problem is complex, it is OK to add more paragraphs. All paragraphs | |
433 | should be wrapped at 72 characters. | |
9cbeb341 | 434 | |
4319fbef | 435 | A paragraph that explains the root cause of the problem. Again, more |
0c48904f | 436 | than one paragraph is OK. |
4319fbef TDS |
437 | |
438 | Finally, one or more paragraphs that explain how the problem is solved. | |
439 | Don't hesitate to explain complex solutions in detail. | |
440 | ||
441 | Signed-off-by: John DOE <[email protected]> | |
9cbeb341 | 442 | |
9cbeb341 VT |
443 | --- |
444 | Changes v2 -> v3: | |
445 | - foo bar (suggested by Jane) | |
446 | - bar buz | |
447 | ||
448 | Changes v1 -> v2: | |
449 | - alpha bravo (suggested by John) | |
450 | - charly delta | |
451 | --------------- | |
452 | ||
453 | Any patch revision should include the version number. The version number | |
454 | is simply composed of the letter +v+ followed by an +integer+ greater or | |
455 | equal to two (i.e. "PATCH v2", "PATCH v3" ...). | |
456 | ||
457 | This can be easily handled with +git format-patch+ by using the option | |
458 | +--subject-prefix+: | |
459 | ||
460 | --------------------- | |
461 | $ git format-patch --subject-prefix "PATCH v4" \ | |
ffc9e468 | 462 | -M -s -o outgoing origin/master |
9cbeb341 VT |
463 | --------------------- |
464 | ||
59fa7b7f FN |
465 | Since git version 1.8.1, you can also use +-v <n>+ (where <n> is the |
466 | version number): | |
467 | ||
468 | --------------------- | |
469 | $ git format-patch -v4 -M -s -o outgoing origin/master | |
470 | --------------------- | |
471 | ||
76bcbae3 VOR |
472 | When you provide a new version of a patch, please mark the old one as |
473 | superseded in http://patchwork.buildroot.org[patchwork]. You need to | |
474 | create an account on http://patchwork.buildroot.org[patchwork] to be | |
475 | able to modify the status of your patches. Note that you can only change | |
476 | the status of patches you submitted yourself, which means the email | |
477 | address you register in http://patchwork.buildroot.org[patchwork] should | |
478 | match the one you use for sending patches to the mailing list. | |
479 | ||
59fa7b7f FN |
480 | You can also add the +--in-reply-to <message-id>+ option when |
481 | submitting a patch to the mailing list. The id of the mail to reply to | |
482 | can be found under the "Message Id" tag on | |
483 | http://patchwork.buildroot.org[patchwork]. The advantage of | |
484 | *in-reply-to* is that patchwork will automatically mark the previous | |
485 | version of the patch as superseded. | |
486 | ||
a79a54a0 | 487 | [[reporting-bugs]] |
569db405 | 488 | === Reporting issues/bugs or getting help |
a79a54a0 | 489 | |
713134fc TDS |
490 | Before reporting any issue, please check in |
491 | xref:community-resources[the mailing list archive] whether someone has | |
492 | already reported and/or fixed a similar problem. | |
a79a54a0 | 493 | |
1c97e35f | 494 | However you choose to report bugs or get help, either by |
713134fc TDS |
495 | opening a bug in the xref:community-resources[bug tracker] or by |
496 | xref:community-resources[sending a mail to the mailing list], there are | |
ecd23535 | 497 | a number of details to provide in order to help people reproduce and |
a79a54a0 SM |
498 | find a solution to the issue. |
499 | ||
ecd23535 | 500 | Try to think as if you were trying to help someone else; in |
a79a54a0 SM |
501 | that case, what would you need? |
502 | ||
503 | Here is a short list of details to provide in such case: | |
504 | ||
505 | * host machine (OS/release) | |
506 | * version of Buildroot | |
507 | * target for which the build fails | |
1c97e35f | 508 | * package(s) for which the build fails |
a79a54a0 SM |
509 | * the command that fails and its output |
510 | * any information you think that may be relevant | |
511 | ||
1c97e35f | 512 | Additionally, you should add the +.config+ file (or if you know how, a |
eddeec8c | 513 | +defconfig+; see xref:customize-store-buildroot-config[]). |
a79a54a0 | 514 | |
ecd23535 | 515 | If some of these details are too large, do not hesitate to use a |
1c97e35f TDS |
516 | pastebin service. Note that not all available pastebin services will |
517 | preserve Unix-style line terminators when downloading raw pastes. | |
518 | Following pastebin services are known to work correctly: | |
519 | - https://gist.github.com/ | |
520 | - http://code.bulix.org/ | |
e2e57d56 | 521 | |
a9dc2de5 | 522 | === Using the runtime tests framework |
e2e57d56 | 523 | |
a9dc2de5 TP |
524 | Buildroot includes a run-time testing framework built upon Python |
525 | scripting and QEMU runtime execution. The goals of the framework are | |
e2e57d56 MW |
526 | the following: |
527 | ||
a9dc2de5 | 528 | * build a well defined Buildroot configuration |
e2e57d56 | 529 | * optionally, verify some properties of the build output |
a9dc2de5 TP |
530 | * optionally, boot the build results under Qemu, and verify that a |
531 | given feature is working as expected | |
e2e57d56 | 532 | |
a9dc2de5 TP |
533 | The entry point to use the runtime tests framework is the |
534 | +support/testing/run-tests+ tool, which has a series of options | |
535 | documented in the tool's help '-h' description. Some common options | |
536 | include setting the download folder, the output folder, keeping build | |
537 | output, and for multiple test cases, you can set the JLEVEL for each. | |
e2e57d56 MW |
538 | |
539 | Here is an example walk through of running a test case. | |
540 | ||
541 | * For a first step, let us see what all the test case options are. The test | |
542 | cases can be listed by executing +support/testing/run-tests -l+. These tests | |
543 | can all be run individually during test development from the console. Both | |
544 | one at a time and selectively as a group of a subset of tests. | |
545 | ||
546 | --------------------- | |
547 | $ support/testing/run-tests -l | |
548 | List of tests | |
549 | test_run (tests.utils.test_check_package.TestCheckPackage) | |
e2e57d56 MW |
550 | test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootMusl) ... ok |
551 | test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc) ... ok | |
552 | test_run (tests.toolchain.test_external.TestExternalToolchainCCache) ... ok | |
553 | test_run (tests.toolchain.test_external.TestExternalToolchainCtngMusl) ... ok | |
554 | test_run (tests.toolchain.test_external.TestExternalToolchainLinaroArm) ... ok | |
555 | test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv4) ... ok | |
556 | test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv5) ... ok | |
557 | test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv7) ... ok | |
558 | [snip] | |
559 | test_run (tests.init.test_systemd.TestInitSystemSystemdRoFull) ... ok | |
560 | test_run (tests.init.test_systemd.TestInitSystemSystemdRoIfupdown) ... ok | |
561 | test_run (tests.init.test_systemd.TestInitSystemSystemdRoNetworkd) ... ok | |
562 | test_run (tests.init.test_systemd.TestInitSystemSystemdRwFull) ... ok | |
563 | test_run (tests.init.test_systemd.TestInitSystemSystemdRwIfupdown) ... ok | |
564 | test_run (tests.init.test_systemd.TestInitSystemSystemdRwNetworkd) ... ok | |
565 | test_run (tests.init.test_busybox.TestInitSystemBusyboxRo) ... ok | |
566 | test_run (tests.init.test_busybox.TestInitSystemBusyboxRoNet) ... ok | |
567 | test_run (tests.init.test_busybox.TestInitSystemBusyboxRw) ... ok | |
568 | test_run (tests.init.test_busybox.TestInitSystemBusyboxRwNet) ... ok | |
569 | ||
570 | Ran 157 tests in 0.021s | |
571 | ||
572 | OK | |
573 | --------------------- | |
574 | ||
a9dc2de5 | 575 | * Then, to run one test case: |
e2e57d56 MW |
576 | |
577 | --------------------- | |
578 | $ support/testing/run-tests -d dl -o output_folder -k tests.init.test_busybox.TestInitSystemBusyboxRw | |
579 | 15:03:26 TestInitSystemBusyboxRw Starting | |
580 | 15:03:28 TestInitSystemBusyboxRw Building | |
581 | 15:08:18 TestInitSystemBusyboxRw Building done | |
582 | 15:08:27 TestInitSystemBusyboxRw Cleaning up | |
583 | . | |
584 | Ran 1 test in 301.140s | |
585 | ||
586 | OK | |
587 | --------------------- | |
588 | ||
a9dc2de5 TP |
589 | The standard output indicates if the test is successful or not. By |
590 | default, the output folder for the test is deleted automatically | |
591 | unless the option +-k+ is passed to *keep* the output directory. | |
e2e57d56 | 592 | |
a9dc2de5 TP |
593 | ==== Creating a test case |
594 | ||
595 | Within the Buildroot repository, the testing framework is organized at the | |
596 | top level in +support/testing/+ by folders of +conf+, +infra+ and +tests+. | |
597 | All the test cases live under the +tests+ folder and are organized in various | |
598 | folders representing the category of test. | |
599 | ||
600 | The best way to get familiar with how to create a test case is to look | |
601 | at a few of the basic file system +support/testing/tests/fs/+ and init | |
602 | +support/testing/tests/init/+ test scripts. Those tests give good | |
603 | examples of a basic tests that include both checking the build | |
604 | results, and doing runtime tests. There are other more advanced cases | |
605 | that use things like nested +br2-external+ folders to provide | |
606 | skeletons and additional packages. | |
607 | ||
608 | Creating a basic test case involves: | |
609 | ||
610 | * Defining a test class that inherits from +infra.basetest.BRTest+ | |
611 | ||
612 | * Defining the +config+ member of the test class, to the Buildroot | |
613 | configuration to build for this test case. It can optionally rely on | |
614 | configuration snippets provided by the runtime test infrastructure: | |
615 | +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ to get a basic | |
616 | architecture/toolchain configuration, and | |
617 | +infra.basetest.MINIMAL_CONFIG+ to not build any filesystem. The | |
618 | advantage of using +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ is that a | |
619 | matching Linux kernel image is provided, which allows to boot the | |
620 | resulting image in Qemu without having to build a Linux kernel image | |
621 | as part of the test case, therefore significant decreasing the build | |
622 | time required for the test case. | |
623 | ||
624 | * Implementing a +def test_run(self):+ function to implement the | |
625 | actual tests to run after the build has completed. They may be tests | |
626 | that verify the build output, by running command on the host using | |
627 | the +run_cmd_on_host()+ helper function. Or they may boot the | |
628 | generated system in Qemu using the +Emulator+ object available as | |
629 | +self.emulator+ in the test case. For example +self.emulator.boot()+ | |
630 | allows to boot the system in Qemu, +self.emulator.login()+ allows to | |
631 | login, +self.emulator.run()+ allows to run shell commands inside | |
632 | Qemu. | |
633 | ||
634 | After creating the test script, add yourself to the +DEVELOPERS+ file to | |
635 | be the maintainer of that test case. | |
636 | ||
637 | ==== Debugging a test case | |
638 | ||
639 | When a test case runs, the +output_folder+ will contain the following: | |
e2e57d56 MW |
640 | |
641 | --------------------- | |
642 | $ ls output_folder/ | |
643 | TestInitSystemBusyboxRw/ | |
644 | TestInitSystemBusyboxRw-build.log | |
645 | TestInitSystemBusyboxRw-run.log | |
646 | --------------------- | |
647 | ||
a9dc2de5 TP |
648 | +TestInitSystemBusyboxRw/+ is the Buildroot output directory, and it |
649 | is preserved only if the +-k+ option is passed. | |
650 | ||
651 | +TestInitSystemBusyboxRw-build.log+ is the log of the Buildroot build. | |
652 | ||
653 | +TestInitSystemBusyboxRw-run.log+ is the log of the Qemu boot and | |
654 | test. This file will only exist if the build was successful and the | |
655 | test case involves booting under Qemu. | |
656 | ||
657 | If you want to manually run Qemu to do manual tests of the build | |
658 | result, the first few lines of +TestInitSystemBusyboxRw-run.log+ | |
659 | contain the Qemu command line to use. | |
660 | ||
661 | You can also make modifications to the current sources inside the | |
662 | +output_folder+ (e.g. for debug purposes) and rerun the standard | |
663 | Buildroot make targets (in order to regenerate the complete image with | |
664 | the new modifications) and then rerun the test. | |
665 | ||
666 | ==== Runtime tests and Gitlab CI | |
667 | ||
668 | All runtime tests are regularly executed by Buildroot Gitlab CI | |
669 | infrastructure, see .gitlab.yml and | |
670 | https://gitlab.com/buildroot.org/buildroot/-/jobs. | |
671 | ||
672 | You can also use Gitlab CI to test your new test cases, or verify that | |
673 | existing tests continue to work after making changes in Buildroot. | |
e2e57d56 | 674 | |
a9dc2de5 TP |
675 | In order to achieve this, you need to create a fork of the Buildroot |
676 | project on Gitlab, and be able to push branches to your Buildroot fork | |
677 | on Gitlab. | |
e2e57d56 | 678 | |
a9dc2de5 TP |
679 | The name of the branch that you push will determine if a Gitlab CI |
680 | pipeline will be triggered or not, and for which test cases. | |
e2e57d56 | 681 | |
a9dc2de5 TP |
682 | In the examples below, the <name> component of the branch name is an |
683 | arbitrary string you choose. | |
e2e57d56 | 684 | |
a9dc2de5 TP |
685 | * To trigger all run-test test case jobs, push a branch that ends with |
686 | +-runtime-tests+: | |
e2e57d56 MW |
687 | |
688 | --------------------- | |
689 | $ git push gitlab HEAD:<name>-runtime-tests | |
690 | --------------------- | |
691 | ||
a9dc2de5 TP |
692 | * To trigger one or several test case jobs, push a branch that ends |
693 | with the complete test case name | |
694 | (+tests.init.test_busybox.TestInitSystemBusyboxRo+) or with the name | |
695 | of a category of tests (+tests.init.test_busybox+): | |
e2e57d56 MW |
696 | |
697 | --------------------- | |
698 | $ git push gitlab HEAD:<name>-<test case name> | |
699 | --------------------- | |
23186356 TP |
700 | |
701 | Example to run one test: | |
702 | ||
703 | --------------------- | |
704 | $ git push gitlab HEAD:foo-tests.init.test_busybox.TestInitSystemBusyboxRo | |
705 | --------------------- | |
706 | ||
707 | Examples to run several tests part of the same group: | |
708 | ||
709 | --------------------- | |
710 | $ git push gitlab HEAD:foo-tests.init.test_busybox | |
711 | $ git push gitlab HEAD:foo-tests.init | |
712 | --------------------- |