Documentation/filesystems/fuse-io-uring.rst

   1 .. SPDX-License-Identifier: GPL-2.0
   2
   3 =======================================
   4 FUSE-over-io-uring design documentation
   5 =======================================
   6
   7 This documentation covers basic details how the fuse
   8 kernel/userspace communication through io-uring is configured
   9 and works. For generic details about FUSE see fuse.rst.
  10
  11 This document also covers the current interface, which is
  12 still in development and might change.
  13
  14 Limitations
  15 ===========
  16 As of now not all requests types are supported through io-uring, userspace
  17 is required to also handle requests through /dev/fuse after io-uring setup
  18 is complete. Specifically notifications (initiated from the daemon side)
  19 and interrupts.
  20
  21 Fuse io-uring configuration
  22 ===========================
  23
  24 Fuse kernel requests are queued through the classical /dev/fuse
  25 read/write interface - until io-uring setup is complete.
  26
  27 In order to set up fuse-over-io-uring fuse-server (user-space)
  28 needs to submit SQEs (opcode = IORING_OP_URING_CMD) to the /dev/fuse
  29 connection file descriptor. Initial submit is with the sub command
  30 FUSE_URING_REQ_REGISTER, which will just register entries to be
  31 available in the kernel.
  32
  33 Once at least one entry per queue is submitted, kernel starts
  34 to enqueue to ring queues.
  35 Note, every CPU core has its own fuse-io-uring queue.
  36 Userspace handles the CQE/fuse-request and submits the result as
  37 subcommand FUSE_URING_REQ_COMMIT_AND_FETCH - kernel completes
  38 the requests and also marks the entry available again. If there are
  39 pending requests waiting the request will be immediately submitted
  40 to the daemon again.
  41
  42 Initial SQE
  43 -----------::
  44
  45  |                                    |  FUSE filesystem daemon
  46  |                                    |
  47  |                                    |  >io_uring_submit()
  48  |                                    |   IORING_OP_URING_CMD /
  49  |                                    |   FUSE_URING_CMD_REGISTER
  50  |                                    |  [wait cqe]
  51  |                                    |   >io_uring_wait_cqe() or
  52  |                                    |   >io_uring_submit_and_wait()
  53  |                                    |
  54  |  >fuse_uring_cmd()                 |
  55  |   >fuse_uring_register()           |
  56
  57
  58 Sending requests with CQEs
  59 --------------------------::
  60
  61  |                                           |  FUSE filesystem daemon
  62  |                                           |  [waiting for CQEs]
  63  |  "rm /mnt/fuse/file"                      |
  64  |                                           |
  65  |  >sys_unlink()                            |
  66  |    >fuse_unlink()                         |
  67  |      [allocate request]                   |
  68  |      >fuse_send_one()                     |
  69  |        ...                                |
  70  |       >fuse_uring_queue_fuse_req          |
  71  |        [queue request on fg queue]        |
  72  |         >fuse_uring_add_req_to_ring_ent() |
  73  |         ...                               |
  74  |          >fuse_uring_copy_to_ring()       |
  75  |          >io_uring_cmd_done()             |
  76  |       >request_wait_answer()              |
  77  |         [sleep on req->waitq]             |
  78  |                                           |  [receives and handles CQE]
  79  |                                           |  [submit result and fetch next]
  80  |                                           |  >io_uring_submit()
  81  |                                           |   IORING_OP_URING_CMD/
  82  |                                           |   FUSE_URING_CMD_COMMIT_AND_FETCH
  83  |  >fuse_uring_cmd()                        |
  84  |   >fuse_uring_commit_fetch()              |
  85  |    >fuse_uring_commit()                   |
  86  |     >fuse_uring_copy_from_ring()          |
  87  |      [ copy the result to the fuse req]   |
  88  |     >fuse_uring_req_end()                 |
  89  |      >fuse_request_end()                  |
  90  |       [wake up req->waitq]                |
  91  |    >fuse_uring_next_fuse_req              |
  92  |       [wait or handle next req]           |
  93  |                                           |
  94  |       [req->waitq woken up]               |
  95  |    <fuse_unlink()                         |
  96  |  <sys_unlink()                            |
  97
  98
  99