Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 4 additions & 27 deletions Documentation/crypto/userspace-if.rst
Original file line number Diff line number Diff line change
Expand Up @@ -328,33 +328,10 @@ CRYPTO_USER_API_RNG_CAVP option:
Zero-Copy Interface
-------------------
Comment on lines 328 to 329

In addition to the send/write/read/recv system call family, the AF_ALG
interface can be accessed with the zero-copy interface of
splice/vmsplice. As the name indicates, the kernel tries to avoid a copy
operation into kernel space.

The zero-copy operation requires data to be aligned at the page
boundary. Non-aligned data can be used as well, but may require more
operations of the kernel which would defeat the speed gains obtained
from the zero-copy interface.

The system-inherent limit for the size of one zero-copy operation is 16
pages. If more data is to be sent to AF_ALG, user space must slice the
input into segments with a maximum size of 16 pages.

Zero-copy can be used with the following code example (a complete
working example is provided with libkcapi):

::

int pipes[2];

pipe(pipes);
/* input data in iov */
vmsplice(pipes[1], iov, iovlen, SPLICE_F_GIFT);
/* opfd is the file descriptor returned from accept() system call */
splice(pipes[0], NULL, opfd, NULL, ret, 0);
read(opfd, out, outlen);
AF_ALG used to have zero-copy support, but it was removed due to it being a
frequent source of vulnerabilities. For backwards compatibility the splice()
and sendfile() system calls are still supported, but the kernel will make an
internal copy of the data before passing it to the crypto code.


Setsockopt Interface
Expand Down
73 changes: 25 additions & 48 deletions crypto/af_alg.c
Original file line number Diff line number Diff line change
Expand Up @@ -975,7 +975,7 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size,
ssize_t plen;

/* use the existing memory in an allocated page */
if (ctx->merge && !(msg->msg_flags & MSG_SPLICE_PAGES)) {
if (ctx->merge) {
sgl = list_entry(ctx->tsgl_list.prev,
struct af_alg_tsgl, list);
sg = sgl->sg + sgl->cur - 1;
Expand Down Expand Up @@ -1019,60 +1019,37 @@ int af_alg_sendmsg(struct socket *sock, struct msghdr *msg, size_t size,
if (sgl->cur)
sg_unmark_end(sg + sgl->cur - 1);

if (msg->msg_flags & MSG_SPLICE_PAGES) {
struct sg_table sgtable = {
.sgl = sg,
.nents = sgl->cur,
.orig_nents = sgl->cur,
};

plen = extract_iter_to_sg(&msg->msg_iter, len, &sgtable,
MAX_SGL_ENTS - sgl->cur, 0);
if (plen < 0) {
err = plen;
do {
struct page *pg;
unsigned int i = sgl->cur;

plen = min_t(size_t, len, PAGE_SIZE);

pg = alloc_page(GFP_KERNEL);
if (!pg) {
err = -ENOMEM;
goto unlock;
}

for (; sgl->cur < sgtable.nents; sgl->cur++)
get_page(sg_page(&sg[sgl->cur]));
sg_assign_page(sg + i, pg);

err = memcpy_from_msg(page_address(sg_page(sg + i)),
msg, plen);
if (err) {
__free_page(sg_page(sg + i));
sg_assign_page(sg + i, NULL);
goto unlock;
}

sg[i].length = plen;
len -= plen;
ctx->used += plen;
copied += plen;
size -= plen;
} else {
do {
struct page *pg;
unsigned int i = sgl->cur;

plen = min_t(size_t, len, PAGE_SIZE);

pg = alloc_page(GFP_KERNEL);
if (!pg) {
err = -ENOMEM;
goto unlock;
}

sg_assign_page(sg + i, pg);

err = memcpy_from_msg(
page_address(sg_page(sg + i)),
msg, plen);
if (err) {
__free_page(sg_page(sg + i));
sg_assign_page(sg + i, NULL);
goto unlock;
}

sg[i].length = plen;
len -= plen;
ctx->used += plen;
copied += plen;
size -= plen;
sgl->cur++;
} while (len && sgl->cur < MAX_SGL_ENTS);

ctx->merge = plen & (PAGE_SIZE - 1);
}
sgl->cur++;
} while (len && sgl->cur < MAX_SGL_ENTS);

ctx->merge = plen & (PAGE_SIZE - 1);

if (!size)
sg_mark_end(sg + sgl->cur - 1);
Expand Down
8 changes: 4 additions & 4 deletions crypto/algif_aead.c
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@
* The following concept of the memory management is used:
*
* The kernel maintains two SGLs, the TX SGL and the RX SGL. The TX SGL is
* filled by user space with the data submitted via sendmsg (maybe with
* MSG_SPLICE_PAGES). Filling up the TX SGL does not cause a crypto operation
* -- the data will only be tracked by the kernel. Upon receipt of one recvmsg
* call, the caller must provide a buffer which is tracked with the RX SGL.
* filled by user space with the data submitted via sendmsg. Filling up the TX
* SGL does not cause a crypto operation -- the data will only be tracked by the
* kernel. Upon receipt of one recvmsg call, the caller must provide a buffer
* which is tracked with the RX SGL.
*
* During the processing of the recvmsg operation, the cipher request is
* allocated and prepared. As part of the recvmsg operation, the processed
Expand Down
Loading