Skip to content

Fix RSA-OAEP to allow zero-length plaintext per RFC 8017 #10012

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MarkAtwood wants to merge 1 commit into
base: master
Choose a base branch
from
+16 −3
Open

Fix RSA-OAEP to allow zero-length plaintext per RFC 8017 #10012

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions wolfcrypt/src/rsa.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3354,7 +3354,15 @@ static int RsaPublicEncryptEx(const byte* in, word32 inLen, byte* out,
RsaPadding padding;
#endif

if (in == NULL || inLen == 0 || out == NULL || key == NULL) {
if (out == NULL || key == NULL) {
return BAD_FUNC_ARG;
}

/* For OAEP padding (RFC 8017, Section 7.1.1), zero-length messages are
* permitted: the spec requires mLen <= k - 2*hLen - 2, and mLen = 0
* satisfies this for all supported key sizes. For other padding types,
* a zero-length input is invalid. */
Comment on lines +3362 to +3364
Comment on lines +3362 to +3364
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment claims mLen=0 "satisfies this for all supported key sizes", but OAEP also requires k > 2*hLen+2 for the chosen hash; small keys (or large hashes) can still be invalid even with mLen=0. Suggest rewording to only state that mLen=0 satisfies the message-length constraint from RFC 8017 step 1b, without implying all key sizes are valid.

Suggested change
* permitted: the spec requires mLen <= k - 2*hLen - 2, and mLen = 0
* satisfies this for all supported key sizes. For other padding types,
* a zero-length input is invalid. */
* permitted with respect to the step 1b message-length check:
* mLen <= k - 2*hLen - 2. For other padding types, a zero-length input
* is invalid. */

Copilot uses AI. Check for mistakes.
if (in == NULL || (inLen == 0 && pad_type != WC_RSA_OAEP_PAD)) {
Comment on lines +3364 to +3365
Comment on lines +3364 to +3365
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This new argument validation still rejects in==NULL even when inLen==0 with OAEP. If the intent is to allow empty OAEP plaintexts, consider permitting a NULL input pointer when inLen==0 (common C API convention and safe here because OAEP padding copies 0 bytes).

Suggested change
* a zero-length input is invalid. */
if (in == NULL || (inLen == 0 && pad_type != WC_RSA_OAEP_PAD)) {
* a zero-length input is invalid. Permit a NULL input pointer only for
* the empty OAEP case, which is safe because no input bytes are read. */
if ((in == NULL && (inLen != 0 || pad_type != WC_RSA_OAEP_PAD)) ||
(inLen == 0 && pad_type != WC_RSA_OAEP_PAD)) {

Copilot uses AI. Check for mistakes.
return BAD_FUNC_ARG;
}

Expand Down Expand Up @@ -3752,8 +3760,13 @@ static int RsaPrivateDecryptEx(const byte* in, word32 inLen, byte* out,
ret = ctMaskSelInt(ctMaskLTE(ret, (int)outLen), ret,
WC_NO_ERR_TRACE(RSA_BUFFER_E));
#ifndef WOLFSSL_RSA_DECRYPT_TO_0_LEN
ret = ctMaskSelInt(ctMaskNotEq(ret, 0), ret,
WC_NO_ERR_TRACE(RSA_BUFFER_E));
/* RFC 8017 Section 7.1.2: OAEP decryption may produce a valid
* zero-length message. Only reject ret==0 for non-OAEP types. */
{
int zeroOk = ctMaskEq(pad_type, WC_RSA_OAEP_PAD);
ret = ctMaskSelInt(ctMaskNotEq(ret, 0) | zeroOk, ret,
Comment on lines +3766 to +3767
Comment on lines +3766 to +3767
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ctMaskEq()/ctMaskNotEq() return a byte mask (0x00/0xFF) and ctMaskSelInt() takes a byte mask. Declaring zeroOk as byte (and keeping the combined mask as byte) would better match the CT helpers and avoid potential compiler warnings about implicit int->byte truncation.

Suggested change
int zeroOk = ctMaskEq(pad_type, WC_RSA_OAEP_PAD);
ret = ctMaskSelInt(ctMaskNotEq(ret, 0) | zeroOk, ret,
byte zeroOk = ctMaskEq(pad_type, WC_RSA_OAEP_PAD);
byte validRetMask = (byte)(ctMaskNotEq(ret, 0) | zeroOk);
ret = ctMaskSelInt(validRetMask, ret,

Copilot uses AI. Check for mistakes.
WC_NO_ERR_TRACE(RSA_BUFFER_E));
Comment on lines +3763 to +3768
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are existing OAEP padding tests (e.g., rsa_oaep_padding_test() in wolfcrypt/test/test.c) but they don’t cover a successful OAEP encrypt/decrypt round-trip of an empty message nor do they assert that invalid OAEP padding yields a negative error (as opposed to a 0-length "success"). Adding those cases would prevent regressions around the ret==0 handling being changed here.

Copilot uses AI. Check for mistakes.
}
Comment on lines +3763 to +3769
Copy link

Copilot AI Apr 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allowing ret==0 for OAEP here makes OAEP padding errors indistinguishable from a valid empty plaintext: RsaUnPad_OAEP() returns 0 on error by setting idx=pkcsBlockLen (see wolfcrypt/src/rsa.c around the "Return 0 data length on error" logic), so invalid OAEP blocks would now be treated as successful decryptions of an empty message. To support empty messages safely, OAEP unpadding needs a separate validity indicator / negative error return (selected constant-time), and this layer should continue to reject the error case even when the message length is 0.

Suggested change
/* RFC 8017 Section 7.1.2: OAEP decryption may produce a valid
* zero-length message. Only reject ret==0 for non-OAEP types. */
{
int zeroOk = ctMaskEq(pad_type, WC_RSA_OAEP_PAD);
ret = ctMaskSelInt(ctMaskNotEq(ret, 0) | zeroOk, ret,
WC_NO_ERR_TRACE(RSA_BUFFER_E));
}
/* Reject zero-length results unless the dedicated build option
* explicitly enables them. In particular, OAEP unpadding may
* return 0 on error, so allowing ret == 0 here would make an
* invalid OAEP block indistinguishable from a valid empty
* plaintext. */
ret = ctMaskSelInt(ctMaskNotEq(ret, 0), ret,
WC_NO_ERR_TRACE(RSA_BUFFER_E));

Copilot uses AI. Check for mistakes.
#endif
#else
if (outLen < (word32)ret)
Expand Down
Loading