]>
Commit | Line | Data |
---|---|---|
1da177e4 LT |
1 | Dynamic DMA mapping |
2 | =================== | |
3 | ||
4 | David S. Miller <[email protected]> | |
5 | Richard Henderson <[email protected]> | |
6 | Jakub Jelinek <[email protected]> | |
7 | ||
8 | This document describes the DMA mapping system in terms of the pci_ | |
9 | API. For a similar API that works for generic devices, see | |
10 | DMA-API.txt. | |
11 | ||
12 | Most of the 64bit platforms have special hardware that translates bus | |
13 | addresses (DMA addresses) into physical addresses. This is similar to | |
14 | how page tables and/or a TLB translates virtual addresses to physical | |
15 | addresses on a CPU. This is needed so that e.g. PCI devices can | |
16 | access with a Single Address Cycle (32bit DMA address) any page in the | |
17 | 64bit physical address space. Previously in Linux those 64bit | |
18 | platforms had to set artificial limits on the maximum RAM size in the | |
19 | system, so that the virt_to_bus() static scheme works (the DMA address | |
20 | translation tables were simply filled on bootup to map each bus | |
21 | address to the physical page __pa(bus_to_virt())). | |
22 | ||
23 | So that Linux can use the dynamic DMA mapping, it needs some help from the | |
24 | drivers, namely it has to take into account that DMA addresses should be | |
25 | mapped only for the time they are actually used and unmapped after the DMA | |
26 | transfer. | |
27 | ||
28 | The following API will work of course even on platforms where no such | |
29 | hardware exists, see e.g. include/asm-i386/pci.h for how it is implemented on | |
30 | top of the virt_to_bus interface. | |
31 | ||
32 | First of all, you should make sure | |
33 | ||
34 | #include <linux/pci.h> | |
35 | ||
36 | is in your driver. This file will obtain for you the definition of the | |
37 | dma_addr_t (which can hold any valid DMA address for the platform) | |
38 | type which should be used everywhere you hold a DMA (bus) address | |
39 | returned from the DMA mapping functions. | |
40 | ||
41 | What memory is DMA'able? | |
42 | ||
43 | The first piece of information you must know is what kernel memory can | |
44 | be used with the DMA mapping facilities. There has been an unwritten | |
45 | set of rules regarding this, and this text is an attempt to finally | |
46 | write them down. | |
47 | ||
48 | If you acquired your memory via the page allocator | |
49 | (i.e. __get_free_page*()) or the generic memory allocators | |
50 | (i.e. kmalloc() or kmem_cache_alloc()) then you may DMA to/from | |
51 | that memory using the addresses returned from those routines. | |
52 | ||
53 | This means specifically that you may _not_ use the memory/addresses | |
54 | returned from vmalloc() for DMA. It is possible to DMA to the | |
55 | _underlying_ memory mapped into a vmalloc() area, but this requires | |
56 | walking page tables to get the physical addresses, and then | |
57 | translating each of those pages back to a kernel address using | |
58 | something like __va(). [ EDIT: Update this when we integrate | |
59 | Gerd Knorr's generic code which does this. ] | |
60 | ||
21440d31 DB |
61 | This rule also means that you may use neither kernel image addresses |
62 | (items in data/text/bss segments), nor module image addresses, nor | |
63 | stack addresses for DMA. These could all be mapped somewhere entirely | |
64 | different than the rest of physical memory. Even if those classes of | |
65 | memory could physically work with DMA, you'd need to ensure the I/O | |
66 | buffers were cacheline-aligned. Without that, you'd see cacheline | |
67 | sharing problems (data corruption) on CPUs with DMA-incoherent caches. | |
68 | (The CPU could write to one word, DMA would write to a different one | |
69 | in the same cache line, and one of them could be overwritten.) | |
1da177e4 LT |
70 | |
71 | Also, this means that you cannot take the return of a kmap() | |
72 | call and DMA to/from that. This is similar to vmalloc(). | |
73 | ||
74 | What about block I/O and networking buffers? The block I/O and | |
75 | networking subsystems make sure that the buffers they use are valid | |
76 | for you to DMA from/to. | |
77 | ||
78 | DMA addressing limitations | |
79 | ||
80 | Does your device have any DMA addressing limitations? For example, is | |
81 | your device only capable of driving the low order 24-bits of address | |
82 | on the PCI bus for SAC DMA transfers? If so, you need to inform the | |
83 | PCI layer of this fact. | |
84 | ||
85 | By default, the kernel assumes that your device can address the full | |
86 | 32-bits in a SAC cycle. For a 64-bit DAC capable device, this needs | |
87 | to be increased. And for a device with limitations, as discussed in | |
88 | the previous paragraph, it needs to be decreased. | |
89 | ||
90 | pci_alloc_consistent() by default will return 32-bit DMA addresses. | |
91 | PCI-X specification requires PCI-X devices to support 64-bit | |
92 | addressing (DAC) for all transactions. And at least one platform (SGI | |
93 | SN2) requires 64-bit consistent allocations to operate correctly when | |
94 | the IO bus is in PCI-X mode. Therefore, like with pci_set_dma_mask(), | |
95 | it's good practice to call pci_set_consistent_dma_mask() to set the | |
96 | appropriate mask even if your device only supports 32-bit DMA | |
97 | (default) and especially if it's a PCI-X device. | |
98 | ||
99 | For correct operation, you must interrogate the PCI layer in your | |
100 | device probe routine to see if the PCI controller on the machine can | |
101 | properly support the DMA addressing limitation your device has. It is | |
102 | good style to do this even if your device holds the default setting, | |
103 | because this shows that you did think about these issues wrt. your | |
104 | device. | |
105 | ||
106 | The query is performed via a call to pci_set_dma_mask(): | |
107 | ||
108 | int pci_set_dma_mask(struct pci_dev *pdev, u64 device_mask); | |
109 | ||
670e9f34 | 110 | The query for consistent allocations is performed via a call to |
1da177e4 LT |
111 | pci_set_consistent_dma_mask(): |
112 | ||
113 | int pci_set_consistent_dma_mask(struct pci_dev *pdev, u64 device_mask); | |
114 | ||
115 | Here, pdev is a pointer to the PCI device struct of your device, and | |
116 | device_mask is a bit mask describing which bits of a PCI address your | |
117 | device supports. It returns zero if your card can perform DMA | |
118 | properly on the machine given the address mask you provided. | |
119 | ||
84eb8d06 | 120 | If it returns non-zero, your device cannot perform DMA properly on |
1da177e4 LT |
121 | this platform, and attempting to do so will result in undefined |
122 | behavior. You must either use a different mask, or not use DMA. | |
123 | ||
124 | This means that in the failure case, you have three options: | |
125 | ||
126 | 1) Use another DMA mask, if possible (see below). | |
127 | 2) Use some non-DMA mode for data transfer, if possible. | |
128 | 3) Ignore this device and do not initialize it. | |
129 | ||
130 | It is recommended that your driver print a kernel KERN_WARNING message | |
131 | when you end up performing either #2 or #3. In this manner, if a user | |
132 | of your driver reports that performance is bad or that the device is not | |
133 | even detected, you can ask them for the kernel messages to find out | |
134 | exactly why. | |
135 | ||
136 | The standard 32-bit addressing PCI device would do something like | |
137 | this: | |
138 | ||
139 | if (pci_set_dma_mask(pdev, DMA_32BIT_MASK)) { | |
140 | printk(KERN_WARNING | |
141 | "mydev: No suitable DMA available.\n"); | |
142 | goto ignore_this_device; | |
143 | } | |
144 | ||
145 | Another common scenario is a 64-bit capable device. The approach | |
146 | here is to try for 64-bit DAC addressing, but back down to a | |
147 | 32-bit mask should that fail. The PCI platform code may fail the | |
148 | 64-bit mask not because the platform is not capable of 64-bit | |
149 | addressing. Rather, it may fail in this case simply because | |
150 | 32-bit SAC addressing is done more efficiently than DAC addressing. | |
151 | Sparc64 is one platform which behaves in this way. | |
152 | ||
153 | Here is how you would handle a 64-bit capable device which can drive | |
154 | all 64-bits when accessing streaming DMA: | |
155 | ||
156 | int using_dac; | |
157 | ||
158 | if (!pci_set_dma_mask(pdev, DMA_64BIT_MASK)) { | |
159 | using_dac = 1; | |
160 | } else if (!pci_set_dma_mask(pdev, DMA_32BIT_MASK)) { | |
161 | using_dac = 0; | |
162 | } else { | |
163 | printk(KERN_WARNING | |
164 | "mydev: No suitable DMA available.\n"); | |
165 | goto ignore_this_device; | |
166 | } | |
167 | ||
168 | If a card is capable of using 64-bit consistent allocations as well, | |
169 | the case would look like this: | |
170 | ||
171 | int using_dac, consistent_using_dac; | |
172 | ||
173 | if (!pci_set_dma_mask(pdev, DMA_64BIT_MASK)) { | |
174 | using_dac = 1; | |
175 | consistent_using_dac = 1; | |
176 | pci_set_consistent_dma_mask(pdev, DMA_64BIT_MASK); | |
177 | } else if (!pci_set_dma_mask(pdev, DMA_32BIT_MASK)) { | |
178 | using_dac = 0; | |
179 | consistent_using_dac = 0; | |
180 | pci_set_consistent_dma_mask(pdev, DMA_32BIT_MASK); | |
181 | } else { | |
182 | printk(KERN_WARNING | |
183 | "mydev: No suitable DMA available.\n"); | |
184 | goto ignore_this_device; | |
185 | } | |
186 | ||
187 | pci_set_consistent_dma_mask() will always be able to set the same or a | |
188 | smaller mask as pci_set_dma_mask(). However for the rare case that a | |
189 | device driver only uses consistent allocations, one would have to | |
190 | check the return value from pci_set_consistent_dma_mask(). | |
191 | ||
192 | If your 64-bit device is going to be an enormous consumer of DMA | |
193 | mappings, this can be problematic since the DMA mappings are a | |
194 | finite resource on many platforms. Please see the "DAC Addressing | |
195 | for Address Space Hungry Devices" section near the end of this | |
196 | document for how to handle this case. | |
197 | ||
198 | Finally, if your device can only drive the low 24-bits of | |
199 | address during PCI bus mastering you might do something like: | |
200 | ||
56b146d3 | 201 | if (pci_set_dma_mask(pdev, DMA_24BIT_MASK)) { |
1da177e4 LT |
202 | printk(KERN_WARNING |
203 | "mydev: 24-bit DMA addressing not available.\n"); | |
204 | goto ignore_this_device; | |
205 | } | |
910638ae MG |
206 | [Better use DMA_24BIT_MASK instead of 0x00ffffff. |
207 | See linux/include/dma-mapping.h for reference.] | |
1da177e4 LT |
208 | |
209 | When pci_set_dma_mask() is successful, and returns zero, the PCI layer | |
210 | saves away this mask you have provided. The PCI layer will use this | |
211 | information later when you make DMA mappings. | |
212 | ||
213 | There is a case which we are aware of at this time, which is worth | |
214 | mentioning in this documentation. If your device supports multiple | |
215 | functions (for example a sound card provides playback and record | |
216 | functions) and the various different functions have _different_ | |
217 | DMA addressing limitations, you may wish to probe each mask and | |
218 | only provide the functionality which the machine can handle. It | |
56b146d3 | 219 | is important that the last call to pci_set_dma_mask() be for the |
1da177e4 LT |
220 | most specific mask. |
221 | ||
222 | Here is pseudo-code showing how this might be done: | |
223 | ||
224 | #define PLAYBACK_ADDRESS_BITS DMA_32BIT_MASK | |
225 | #define RECORD_ADDRESS_BITS 0x00ffffff | |
226 | ||
227 | struct my_sound_card *card; | |
228 | struct pci_dev *pdev; | |
229 | ||
230 | ... | |
231 | if (!pci_set_dma_mask(pdev, PLAYBACK_ADDRESS_BITS)) { | |
232 | card->playback_enabled = 1; | |
233 | } else { | |
234 | card->playback_enabled = 0; | |
235 | printk(KERN_WARN "%s: Playback disabled due to DMA limitations.\n", | |
236 | card->name); | |
237 | } | |
238 | if (!pci_set_dma_mask(pdev, RECORD_ADDRESS_BITS)) { | |
239 | card->record_enabled = 1; | |
240 | } else { | |
241 | card->record_enabled = 0; | |
242 | printk(KERN_WARN "%s: Record disabled due to DMA limitations.\n", | |
243 | card->name); | |
244 | } | |
245 | ||
246 | A sound card was used as an example here because this genre of PCI | |
247 | devices seems to be littered with ISA chips given a PCI front end, | |
248 | and thus retaining the 16MB DMA addressing limitations of ISA. | |
249 | ||
250 | Types of DMA mappings | |
251 | ||
252 | There are two types of DMA mappings: | |
253 | ||
254 | - Consistent DMA mappings which are usually mapped at driver | |
255 | initialization, unmapped at the end and for which the hardware should | |
256 | guarantee that the device and the CPU can access the data | |
257 | in parallel and will see updates made by each other without any | |
258 | explicit software flushing. | |
259 | ||
260 | Think of "consistent" as "synchronous" or "coherent". | |
261 | ||
262 | The current default is to return consistent memory in the low 32 | |
263 | bits of the PCI bus space. However, for future compatibility you | |
264 | should set the consistent mask even if this default is fine for your | |
265 | driver. | |
266 | ||
267 | Good examples of what to use consistent mappings for are: | |
268 | ||
269 | - Network card DMA ring descriptors. | |
270 | - SCSI adapter mailbox command data structures. | |
271 | - Device firmware microcode executed out of | |
272 | main memory. | |
273 | ||
274 | The invariant these examples all require is that any CPU store | |
275 | to memory is immediately visible to the device, and vice | |
276 | versa. Consistent mappings guarantee this. | |
277 | ||
278 | IMPORTANT: Consistent DMA memory does not preclude the usage of | |
279 | proper memory barriers. The CPU may reorder stores to | |
280 | consistent memory just as it may normal memory. Example: | |
281 | if it is important for the device to see the first word | |
282 | of a descriptor updated before the second, you must do | |
283 | something like: | |
284 | ||
285 | desc->word0 = address; | |
286 | wmb(); | |
287 | desc->word1 = DESC_VALID; | |
288 | ||
289 | in order to get correct behavior on all platforms. | |
290 | ||
21440d31 DB |
291 | Also, on some platforms your driver may need to flush CPU write |
292 | buffers in much the same way as it needs to flush write buffers | |
293 | found in PCI bridges (such as by reading a register's value | |
294 | after writing it). | |
295 | ||
1da177e4 LT |
296 | - Streaming DMA mappings which are usually mapped for one DMA transfer, |
297 | unmapped right after it (unless you use pci_dma_sync_* below) and for which | |
298 | hardware can optimize for sequential accesses. | |
299 | ||
300 | This of "streaming" as "asynchronous" or "outside the coherency | |
301 | domain". | |
302 | ||
303 | Good examples of what to use streaming mappings for are: | |
304 | ||
305 | - Networking buffers transmitted/received by a device. | |
306 | - Filesystem buffers written/read by a SCSI device. | |
307 | ||
308 | The interfaces for using this type of mapping were designed in | |
309 | such a way that an implementation can make whatever performance | |
310 | optimizations the hardware allows. To this end, when using | |
311 | such mappings you must be explicit about what you want to happen. | |
312 | ||
313 | Neither type of DMA mapping has alignment restrictions that come | |
314 | from PCI, although some devices may have such restrictions. | |
21440d31 DB |
315 | Also, systems with caches that aren't DMA-coherent will work better |
316 | when the underlying buffers don't share cache lines with other data. | |
317 | ||
1da177e4 LT |
318 | |
319 | Using Consistent DMA mappings. | |
320 | ||
321 | To allocate and map large (PAGE_SIZE or so) consistent DMA regions, | |
322 | you should do: | |
323 | ||
324 | dma_addr_t dma_handle; | |
325 | ||
326 | cpu_addr = pci_alloc_consistent(dev, size, &dma_handle); | |
327 | ||
328 | where dev is a struct pci_dev *. You should pass NULL for PCI like buses | |
329 | where devices don't have struct pci_dev (like ISA, EISA). This may be | |
330 | called in interrupt context. | |
331 | ||
332 | This argument is needed because the DMA translations may be bus | |
333 | specific (and often is private to the bus which the device is attached | |
334 | to). | |
335 | ||
336 | Size is the length of the region you want to allocate, in bytes. | |
337 | ||
338 | This routine will allocate RAM for that region, so it acts similarly to | |
339 | __get_free_pages (but takes size instead of a page order). If your | |
340 | driver needs regions sized smaller than a page, you may prefer using | |
341 | the pci_pool interface, described below. | |
342 | ||
343 | The consistent DMA mapping interfaces, for non-NULL dev, will by | |
344 | default return a DMA address which is SAC (Single Address Cycle) | |
345 | addressable. Even if the device indicates (via PCI dma mask) that it | |
346 | may address the upper 32-bits and thus perform DAC cycles, consistent | |
347 | allocation will only return > 32-bit PCI addresses for DMA if the | |
348 | consistent dma mask has been explicitly changed via | |
349 | pci_set_consistent_dma_mask(). This is true of the pci_pool interface | |
350 | as well. | |
351 | ||
352 | pci_alloc_consistent returns two values: the virtual address which you | |
353 | can use to access it from the CPU and dma_handle which you pass to the | |
354 | card. | |
355 | ||
356 | The cpu return address and the DMA bus master address are both | |
357 | guaranteed to be aligned to the smallest PAGE_SIZE order which | |
358 | is greater than or equal to the requested size. This invariant | |
359 | exists (for example) to guarantee that if you allocate a chunk | |
360 | which is smaller than or equal to 64 kilobytes, the extent of the | |
361 | buffer you receive will not cross a 64K boundary. | |
362 | ||
363 | To unmap and free such a DMA region, you call: | |
364 | ||
365 | pci_free_consistent(dev, size, cpu_addr, dma_handle); | |
366 | ||
367 | where dev, size are the same as in the above call and cpu_addr and | |
368 | dma_handle are the values pci_alloc_consistent returned to you. | |
369 | This function may not be called in interrupt context. | |
370 | ||
371 | If your driver needs lots of smaller memory regions, you can write | |
372 | custom code to subdivide pages returned by pci_alloc_consistent, | |
373 | or you can use the pci_pool API to do that. A pci_pool is like | |
374 | a kmem_cache, but it uses pci_alloc_consistent not __get_free_pages. | |
375 | Also, it understands common hardware constraints for alignment, | |
376 | like queue heads needing to be aligned on N byte boundaries. | |
377 | ||
378 | Create a pci_pool like this: | |
379 | ||
380 | struct pci_pool *pool; | |
381 | ||
382 | pool = pci_pool_create(name, dev, size, align, alloc); | |
383 | ||
384 | The "name" is for diagnostics (like a kmem_cache name); dev and size | |
385 | are as above. The device's hardware alignment requirement for this | |
386 | type of data is "align" (which is expressed in bytes, and must be a | |
387 | power of two). If your device has no boundary crossing restrictions, | |
388 | pass 0 for alloc; passing 4096 says memory allocated from this pool | |
389 | must not cross 4KByte boundaries (but at that time it may be better to | |
390 | go for pci_alloc_consistent directly instead). | |
391 | ||
392 | Allocate memory from a pci pool like this: | |
393 | ||
394 | cpu_addr = pci_pool_alloc(pool, flags, &dma_handle); | |
395 | ||
396 | flags are SLAB_KERNEL if blocking is permitted (not in_interrupt nor | |
397 | holding SMP locks), SLAB_ATOMIC otherwise. Like pci_alloc_consistent, | |
398 | this returns two values, cpu_addr and dma_handle. | |
399 | ||
400 | Free memory that was allocated from a pci_pool like this: | |
401 | ||
402 | pci_pool_free(pool, cpu_addr, dma_handle); | |
403 | ||
404 | where pool is what you passed to pci_pool_alloc, and cpu_addr and | |
405 | dma_handle are the values pci_pool_alloc returned. This function | |
406 | may be called in interrupt context. | |
407 | ||
408 | Destroy a pci_pool by calling: | |
409 | ||
410 | pci_pool_destroy(pool); | |
411 | ||
412 | Make sure you've called pci_pool_free for all memory allocated | |
413 | from a pool before you destroy the pool. This function may not | |
414 | be called in interrupt context. | |
415 | ||
416 | DMA Direction | |
417 | ||
418 | The interfaces described in subsequent portions of this document | |
419 | take a DMA direction argument, which is an integer and takes on | |
420 | one of the following values: | |
421 | ||
422 | PCI_DMA_BIDIRECTIONAL | |
423 | PCI_DMA_TODEVICE | |
424 | PCI_DMA_FROMDEVICE | |
425 | PCI_DMA_NONE | |
426 | ||
427 | One should provide the exact DMA direction if you know it. | |
428 | ||
429 | PCI_DMA_TODEVICE means "from main memory to the PCI device" | |
430 | PCI_DMA_FROMDEVICE means "from the PCI device to main memory" | |
431 | It is the direction in which the data moves during the DMA | |
432 | transfer. | |
433 | ||
434 | You are _strongly_ encouraged to specify this as precisely | |
435 | as you possibly can. | |
436 | ||
437 | If you absolutely cannot know the direction of the DMA transfer, | |
438 | specify PCI_DMA_BIDIRECTIONAL. It means that the DMA can go in | |
439 | either direction. The platform guarantees that you may legally | |
440 | specify this, and that it will work, but this may be at the | |
441 | cost of performance for example. | |
442 | ||
443 | The value PCI_DMA_NONE is to be used for debugging. One can | |
444 | hold this in a data structure before you come to know the | |
445 | precise direction, and this will help catch cases where your | |
446 | direction tracking logic has failed to set things up properly. | |
447 | ||
448 | Another advantage of specifying this value precisely (outside of | |
449 | potential platform-specific optimizations of such) is for debugging. | |
450 | Some platforms actually have a write permission boolean which DMA | |
451 | mappings can be marked with, much like page protections in the user | |
452 | program address space. Such platforms can and do report errors in the | |
453 | kernel logs when the PCI controller hardware detects violation of the | |
454 | permission setting. | |
455 | ||
456 | Only streaming mappings specify a direction, consistent mappings | |
457 | implicitly have a direction attribute setting of | |
458 | PCI_DMA_BIDIRECTIONAL. | |
459 | ||
be7db055 CH |
460 | The SCSI subsystem tells you the direction to use in the |
461 | 'sc_data_direction' member of the SCSI command your driver is | |
462 | working on. | |
1da177e4 LT |
463 | |
464 | For Networking drivers, it's a rather simple affair. For transmit | |
465 | packets, map/unmap them with the PCI_DMA_TODEVICE direction | |
466 | specifier. For receive packets, just the opposite, map/unmap them | |
467 | with the PCI_DMA_FROMDEVICE direction specifier. | |
468 | ||
469 | Using Streaming DMA mappings | |
470 | ||
471 | The streaming DMA mapping routines can be called from interrupt | |
472 | context. There are two versions of each map/unmap, one which will | |
473 | map/unmap a single memory region, and one which will map/unmap a | |
474 | scatterlist. | |
475 | ||
476 | To map a single region, you do: | |
477 | ||
478 | struct pci_dev *pdev = mydev->pdev; | |
479 | dma_addr_t dma_handle; | |
480 | void *addr = buffer->ptr; | |
481 | size_t size = buffer->len; | |
482 | ||
483 | dma_handle = pci_map_single(dev, addr, size, direction); | |
484 | ||
485 | and to unmap it: | |
486 | ||
487 | pci_unmap_single(dev, dma_handle, size, direction); | |
488 | ||
489 | You should call pci_unmap_single when the DMA activity is finished, e.g. | |
490 | from the interrupt which told you that the DMA transfer is done. | |
491 | ||
492 | Using cpu pointers like this for single mappings has a disadvantage, | |
493 | you cannot reference HIGHMEM memory in this way. Thus, there is a | |
494 | map/unmap interface pair akin to pci_{map,unmap}_single. These | |
495 | interfaces deal with page/offset pairs instead of cpu pointers. | |
496 | Specifically: | |
497 | ||
498 | struct pci_dev *pdev = mydev->pdev; | |
499 | dma_addr_t dma_handle; | |
500 | struct page *page = buffer->page; | |
501 | unsigned long offset = buffer->offset; | |
502 | size_t size = buffer->len; | |
503 | ||
504 | dma_handle = pci_map_page(dev, page, offset, size, direction); | |
505 | ||
506 | ... | |
507 | ||
508 | pci_unmap_page(dev, dma_handle, size, direction); | |
509 | ||
510 | Here, "offset" means byte offset within the given page. | |
511 | ||
512 | With scatterlists, you map a region gathered from several regions by: | |
513 | ||
514 | int i, count = pci_map_sg(dev, sglist, nents, direction); | |
515 | struct scatterlist *sg; | |
516 | ||
517 | for (i = 0, sg = sglist; i < count; i++, sg++) { | |
518 | hw_address[i] = sg_dma_address(sg); | |
519 | hw_len[i] = sg_dma_len(sg); | |
520 | } | |
521 | ||
522 | where nents is the number of entries in the sglist. | |
523 | ||
524 | The implementation is free to merge several consecutive sglist entries | |
525 | into one (e.g. if DMA mapping is done with PAGE_SIZE granularity, any | |
526 | consecutive sglist entries can be merged into one provided the first one | |
527 | ends and the second one starts on a page boundary - in fact this is a huge | |
528 | advantage for cards which either cannot do scatter-gather or have very | |
529 | limited number of scatter-gather entries) and returns the actual number | |
530 | of sg entries it mapped them to. On failure 0 is returned. | |
531 | ||
532 | Then you should loop count times (note: this can be less than nents times) | |
533 | and use sg_dma_address() and sg_dma_len() macros where you previously | |
534 | accessed sg->address and sg->length as shown above. | |
535 | ||
536 | To unmap a scatterlist, just call: | |
537 | ||
538 | pci_unmap_sg(dev, sglist, nents, direction); | |
539 | ||
540 | Again, make sure DMA activity has already finished. | |
541 | ||
542 | PLEASE NOTE: The 'nents' argument to the pci_unmap_sg call must be | |
543 | the _same_ one you passed into the pci_map_sg call, | |
544 | it should _NOT_ be the 'count' value _returned_ from the | |
545 | pci_map_sg call. | |
546 | ||
547 | Every pci_map_{single,sg} call should have its pci_unmap_{single,sg} | |
548 | counterpart, because the bus address space is a shared resource (although | |
549 | in some ports the mapping is per each BUS so less devices contend for the | |
550 | same bus address space) and you could render the machine unusable by eating | |
551 | all bus addresses. | |
552 | ||
553 | If you need to use the same streaming DMA region multiple times and touch | |
554 | the data in between the DMA transfers, the buffer needs to be synced | |
555 | properly in order for the cpu and device to see the most uptodate and | |
556 | correct copy of the DMA buffer. | |
557 | ||
558 | So, firstly, just map it with pci_map_{single,sg}, and after each DMA | |
559 | transfer call either: | |
560 | ||
561 | pci_dma_sync_single_for_cpu(dev, dma_handle, size, direction); | |
562 | ||
563 | or: | |
564 | ||
565 | pci_dma_sync_sg_for_cpu(dev, sglist, nents, direction); | |
566 | ||
567 | as appropriate. | |
568 | ||
569 | Then, if you wish to let the device get at the DMA area again, | |
570 | finish accessing the data with the cpu, and then before actually | |
571 | giving the buffer to the hardware call either: | |
572 | ||
573 | pci_dma_sync_single_for_device(dev, dma_handle, size, direction); | |
574 | ||
575 | or: | |
576 | ||
577 | pci_dma_sync_sg_for_device(dev, sglist, nents, direction); | |
578 | ||
579 | as appropriate. | |
580 | ||
581 | After the last DMA transfer call one of the DMA unmap routines | |
582 | pci_unmap_{single,sg}. If you don't touch the data from the first pci_map_* | |
583 | call till pci_unmap_*, then you don't have to call the pci_dma_sync_* | |
584 | routines at all. | |
585 | ||
586 | Here is pseudo code which shows a situation in which you would need | |
587 | to use the pci_dma_sync_*() interfaces. | |
588 | ||
589 | my_card_setup_receive_buffer(struct my_card *cp, char *buffer, int len) | |
590 | { | |
591 | dma_addr_t mapping; | |
592 | ||
593 | mapping = pci_map_single(cp->pdev, buffer, len, PCI_DMA_FROMDEVICE); | |
594 | ||
595 | cp->rx_buf = buffer; | |
596 | cp->rx_len = len; | |
597 | cp->rx_dma = mapping; | |
598 | ||
599 | give_rx_buf_to_card(cp); | |
600 | } | |
601 | ||
602 | ... | |
603 | ||
604 | my_card_interrupt_handler(int irq, void *devid, struct pt_regs *regs) | |
605 | { | |
606 | struct my_card *cp = devid; | |
607 | ||
608 | ... | |
609 | if (read_card_status(cp) == RX_BUF_TRANSFERRED) { | |
610 | struct my_card_header *hp; | |
611 | ||
612 | /* Examine the header to see if we wish | |
613 | * to accept the data. But synchronize | |
614 | * the DMA transfer with the CPU first | |
615 | * so that we see updated contents. | |
616 | */ | |
617 | pci_dma_sync_single_for_cpu(cp->pdev, cp->rx_dma, | |
618 | cp->rx_len, | |
619 | PCI_DMA_FROMDEVICE); | |
620 | ||
621 | /* Now it is safe to examine the buffer. */ | |
622 | hp = (struct my_card_header *) cp->rx_buf; | |
623 | if (header_is_ok(hp)) { | |
624 | pci_unmap_single(cp->pdev, cp->rx_dma, cp->rx_len, | |
625 | PCI_DMA_FROMDEVICE); | |
626 | pass_to_upper_layers(cp->rx_buf); | |
627 | make_and_setup_new_rx_buf(cp); | |
628 | } else { | |
629 | /* Just sync the buffer and give it back | |
630 | * to the card. | |
631 | */ | |
632 | pci_dma_sync_single_for_device(cp->pdev, | |
633 | cp->rx_dma, | |
634 | cp->rx_len, | |
635 | PCI_DMA_FROMDEVICE); | |
636 | give_rx_buf_to_card(cp); | |
637 | } | |
638 | } | |
639 | } | |
640 | ||
641 | Drivers converted fully to this interface should not use virt_to_bus any | |
642 | longer, nor should they use bus_to_virt. Some drivers have to be changed a | |
643 | little bit, because there is no longer an equivalent to bus_to_virt in the | |
644 | dynamic DMA mapping scheme - you have to always store the DMA addresses | |
645 | returned by the pci_alloc_consistent, pci_pool_alloc, and pci_map_single | |
646 | calls (pci_map_sg stores them in the scatterlist itself if the platform | |
647 | supports dynamic DMA mapping in hardware) in your driver structures and/or | |
648 | in the card registers. | |
649 | ||
650 | All PCI drivers should be using these interfaces with no exceptions. | |
651 | It is planned to completely remove virt_to_bus() and bus_to_virt() as | |
652 | they are entirely deprecated. Some ports already do not provide these | |
653 | as it is impossible to correctly support them. | |
654 | ||
655 | 64-bit DMA and DAC cycle support | |
656 | ||
657 | Do you understand all of the text above? Great, then you already | |
658 | know how to use 64-bit DMA addressing under Linux. Simply make | |
659 | the appropriate pci_set_dma_mask() calls based upon your cards | |
660 | capabilities, then use the mapping APIs above. | |
661 | ||
662 | It is that simple. | |
663 | ||
664 | Well, not for some odd devices. See the next section for information | |
665 | about that. | |
666 | ||
1da177e4 LT |
667 | Optimizing Unmap State Space Consumption |
668 | ||
669 | On many platforms, pci_unmap_{single,page}() is simply a nop. | |
670 | Therefore, keeping track of the mapping address and length is a waste | |
671 | of space. Instead of filling your drivers up with ifdefs and the like | |
672 | to "work around" this (which would defeat the whole purpose of a | |
673 | portable API) the following facilities are provided. | |
674 | ||
675 | Actually, instead of describing the macros one by one, we'll | |
676 | transform some example code. | |
677 | ||
678 | 1) Use DECLARE_PCI_UNMAP_{ADDR,LEN} in state saving structures. | |
679 | Example, before: | |
680 | ||
681 | struct ring_state { | |
682 | struct sk_buff *skb; | |
683 | dma_addr_t mapping; | |
684 | __u32 len; | |
685 | }; | |
686 | ||
687 | after: | |
688 | ||
689 | struct ring_state { | |
690 | struct sk_buff *skb; | |
691 | DECLARE_PCI_UNMAP_ADDR(mapping) | |
692 | DECLARE_PCI_UNMAP_LEN(len) | |
693 | }; | |
694 | ||
695 | NOTE: DO NOT put a semicolon at the end of the DECLARE_*() | |
696 | macro. | |
697 | ||
698 | 2) Use pci_unmap_{addr,len}_set to set these values. | |
699 | Example, before: | |
700 | ||
701 | ringp->mapping = FOO; | |
702 | ringp->len = BAR; | |
703 | ||
704 | after: | |
705 | ||
706 | pci_unmap_addr_set(ringp, mapping, FOO); | |
707 | pci_unmap_len_set(ringp, len, BAR); | |
708 | ||
709 | 3) Use pci_unmap_{addr,len} to access these values. | |
710 | Example, before: | |
711 | ||
712 | pci_unmap_single(pdev, ringp->mapping, ringp->len, | |
713 | PCI_DMA_FROMDEVICE); | |
714 | ||
715 | after: | |
716 | ||
717 | pci_unmap_single(pdev, | |
718 | pci_unmap_addr(ringp, mapping), | |
719 | pci_unmap_len(ringp, len), | |
720 | PCI_DMA_FROMDEVICE); | |
721 | ||
722 | It really should be self-explanatory. We treat the ADDR and LEN | |
723 | separately, because it is possible for an implementation to only | |
724 | need the address in order to perform the unmap operation. | |
725 | ||
726 | Platform Issues | |
727 | ||
728 | If you are just writing drivers for Linux and do not maintain | |
729 | an architecture port for the kernel, you can safely skip down | |
730 | to "Closing". | |
731 | ||
732 | 1) Struct scatterlist requirements. | |
733 | ||
734 | Struct scatterlist must contain, at a minimum, the following | |
735 | members: | |
736 | ||
737 | struct page *page; | |
738 | unsigned int offset; | |
739 | unsigned int length; | |
740 | ||
741 | The base address is specified by a "page+offset" pair. | |
742 | ||
743 | Previous versions of struct scatterlist contained a "void *address" | |
744 | field that was sometimes used instead of page+offset. As of Linux | |
745 | 2.5., page+offset is always used, and the "address" field has been | |
746 | deleted. | |
747 | ||
748 | 2) More to come... | |
749 | ||
750 | Handling Errors | |
751 | ||
752 | DMA address space is limited on some architectures and an allocation | |
753 | failure can be determined by: | |
754 | ||
755 | - checking if pci_alloc_consistent returns NULL or pci_map_sg returns 0 | |
756 | ||
757 | - checking the returned dma_addr_t of pci_map_single and pci_map_page | |
758 | by using pci_dma_mapping_error(): | |
759 | ||
760 | dma_addr_t dma_handle; | |
761 | ||
762 | dma_handle = pci_map_single(dev, addr, size, direction); | |
763 | if (pci_dma_mapping_error(dma_handle)) { | |
764 | /* | |
765 | * reduce current DMA mapping usage, | |
766 | * delay and try again later or | |
767 | * reset driver. | |
768 | */ | |
769 | } | |
770 | ||
771 | Closing | |
772 | ||
773 | This document, and the API itself, would not be in it's current | |
774 | form without the feedback and suggestions from numerous individuals. | |
775 | We would like to specifically mention, in no particular order, the | |
776 | following people: | |
777 | ||
778 | Russell King <[email protected]> | |
779 | Leo Dagum <[email protected]> | |
780 | Ralf Baechle <[email protected]> | |
781 | Grant Grundler <[email protected]> | |
782 | Jay Estabrook <[email protected]> | |
783 | Thomas Sailer <[email protected]> | |
784 | Andrea Arcangeli <[email protected]> | |
785 | Jens Axboe <[email protected]> | |
786 | David Mosberger-Tang <[email protected]> |