gh-73613: Support Base32 and Base64 without padding

Doc/library/base64.rst

@@ -51,7 +51,7 @@ The :rfc:`4648` encodings are suitable for encoding binary data so that it can b
 safely sent by email, used as parts of URLs, or included as part of an HTTP
 POST request.
 
-.. function:: b64encode(s, altchars=None, *, wrapcol=0)
+.. function:: b64encode(s, altchars=None, *, padded=True, wrapcol=0)
 
    Encode the :term:`bytes-like object` *s* using Base64 and return the encoded
    :class:`bytes`.
@@ -61,6 +61,10 @@ POST request.
    This allows an application to e.g. generate URL or filesystem safe Base64
    strings.  The default is ``None``, for which the standard Base64 alphabet is used.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 4.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not insert any newlines.
@@ -69,11 +73,11 @@ POST request.
    :exc:`TypeError` if *altchars* is not a :term:`bytes-like object`.
 
    .. versionchanged:: 3.15
-      Added the *wrapcol* parameter.
+      Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b64decode(s, altchars=None, validate=False)
-              b64decode(s, altchars=None, validate=True, *, ignorechars)
+.. function:: b64decode(s, altchars=None, validate=False, *, padded=True)
+              b64decode(s, altchars=None, validate=True, *, ignorechars, padded=True)
 
    Decode the Base64 encoded :term:`bytes-like object` or ASCII string
    *s* and return the decoded :class:`bytes`.
@@ -82,6 +86,11 @@ POST request.
    of length 2 which specifies the alternative alphabet used instead of the
    ``+`` and ``/`` characters.
 
+   If *padded* is true, the last group of 4 base 64 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *validate* and *ignorechars*).
+
    A :exc:`binascii.Error` exception is raised
    if *s* is incorrectly padded.
 
@@ -106,7 +115,7 @@ POST request.
    For more information about the strict base64 check, see :func:`binascii.a2b_base64`
 
    .. versionchanged:: 3.15
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *padded* parameters.
 
    .. deprecated:: 3.15
       Accepting the ``+`` and ``/`` characters with an alternative alphabet
@@ -125,41 +134,52 @@ POST request.
    Base64 alphabet and return the decoded :class:`bytes`.
 
 
-.. function:: urlsafe_b64encode(s)
+.. function:: urlsafe_b64encode(s, *, padded=True)
 
    Encode :term:`bytes-like object` *s* using the
    URL- and filesystem-safe alphabet, which
    substitutes ``-`` instead of ``+`` and ``_`` instead of ``/`` in the
    standard Base64 alphabet, and return the encoded :class:`bytes`.  The result
-   can still contain ``=``.
+   can still contain ``=`` if *padded* is true (default).
+
+   .. versionchanged:: next
+      Added the *padded* parameter.
 
 
-.. function:: urlsafe_b64decode(s)
+.. function:: urlsafe_b64decode(s, *, padded=False)
 
    Decode :term:`bytes-like object` or ASCII string *s*
    using the URL- and filesystem-safe
    alphabet, which substitutes ``-`` instead of ``+`` and ``_`` instead of
    ``/`` in the standard Base64 alphabet, and return the decoded
    :class:`bytes`.
 
+   .. versionchanged:: next
+      Added the *padded* parameter.
+      Padding of input is no longer required by default.
+
    .. deprecated:: 3.15
       Accepting the ``+`` and ``/`` characters is now deprecated.
 
 
-.. function:: b32encode(s, *, wrapcol=0)
+.. function:: b32encode(s, *, padded=True, wrapcol=0)
 
    Encode the :term:`bytes-like object` *s* using Base32 and return the
    encoded :class:`bytes`.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 8.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not add any newlines.
 
    .. versionchanged:: next
-      Added the *wrapcol* parameter.
+      Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32decode(s, casefold=False, map01=None, *, ignorechars=b'')
+.. function:: b32decode(s, casefold=False, map01=None, *, padded=True, ignorechars=b'')
 
    Decode the Base32 encoded :term:`bytes-like object` or ASCII string *s* and
    return the decoded :class:`bytes`.
@@ -175,6 +195,11 @@ POST request.
    digit 0 is always mapped to the letter O).  For security purposes the default is
    ``None``, so that 0 and 1 are not allowed in the input.
 
+   If *padded* is true, the last group of 8 base 32 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *ignorechars*).
+
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
 
@@ -183,21 +208,21 @@ POST request.
    input.
 
    .. versionchanged:: next
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *padded* parameters.
 
 
-.. function:: b32hexencode(s, *, wrapcol=0)
+.. function:: b32hexencode(s, *, padded=True, wrapcol=0)
 
    Similar to :func:`b32encode` but uses the Extended Hex Alphabet, as defined in
    :rfc:`4648`.
 
    .. versionadded:: 3.10
 
    .. versionchanged:: next
-      Added the *wrapcol* parameter.
+      Added the *padded* and *wrapcol* parameters.
 
 
-.. function:: b32hexdecode(s, casefold=False, *, ignorechars=b'')
+.. function:: b32hexdecode(s, casefold=False, *, padded=True, ignorechars=b'')
 
    Similar to :func:`b32decode` but uses the Extended Hex Alphabet, as defined in
    :rfc:`4648`.
@@ -210,7 +235,7 @@ POST request.
    .. versionadded:: 3.10
 
    .. versionchanged:: next
-      Added the *ignorechars* parameter.
+      Added the *ignorechars* and *padded* parameters.
 
 
 .. function:: b16encode(s, *, wrapcol=0)

Doc/library/binascii.rst

@@ -48,15 +48,20 @@ The :mod:`!binascii` module defines the following functions:
       Added the *backtick* parameter.
 
 
-.. function:: a2b_base64(string, /, *, alphabet=BASE64_ALPHABET, strict_mode=False)
-              a2b_base64(string, /, *, ignorechars, alphabet=BASE64_ALPHABET, strict_mode=True)
+.. function:: a2b_base64(string, /, *, padded=True, alphabet=BASE64_ALPHABET, strict_mode=False)
+              a2b_base64(string, /, *, ignorechars, padded=True, alphabet=BASE64_ALPHABET, strict_mode=True)
 
    Convert a block of base64 data back to binary and return the binary data. More
    than one line may be passed at a time.
 
    Optional *alphabet* must be a :class:`bytes` object of length 64 which
    specifies an alternative alphabet.
 
+   If *padded* is true, the last group of 4 base 64 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *strict_mode* and *ignorechars*).
+
    If *ignorechars* is specified, it should be a :term:`bytes-like object`
    containing characters to ignore from the input when *strict_mode* is true.
    If *ignorechars* contains the pad character ``'='``,  the pad characters
@@ -79,14 +84,18 @@ The :mod:`!binascii` module defines the following functions:
       Added the *strict_mode* parameter.
 
    .. versionchanged:: 3.15
-      Added the *alphabet* and *ignorechars* parameters.
+      Added the *alphabet*, *ignorechars* and *padded* parameters.
 
 
-.. function:: b2a_base64(data, *, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
+.. function:: b2a_base64(data, *, padded=True, alphabet=BASE64_ALPHABET, wrapcol=0, newline=True)
 
    Convert binary data to a line(s) of ASCII characters in base64 coding,
    as specified in :rfc:`4648`.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 4.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not insert any newlines.
@@ -98,7 +107,7 @@ The :mod:`!binascii` module defines the following functions:
       Added the *newline* parameter.
 
    .. versionchanged:: 3.15
-      Added the *alphabet* and *wrapcol* parameters.
+      Added the *alphabet*, *padded* and *wrapcol* parameters.
 
 
 .. function:: a2b_ascii85(string, /, *, foldspaces=False, adobe=False, ignorechars=b'')
@@ -190,7 +199,7 @@ The :mod:`!binascii` module defines the following functions:
    .. versionadded:: 3.15
 
 
-.. function:: a2b_base32(string, /, *, alphabet=BASE32_ALPHABET, ignorechars=b'')
+.. function:: a2b_base32(string, /, *, padded=True, alphabet=BASE32_ALPHABET, ignorechars=b'')
 
    Convert base32 data back to binary and return the binary data.
 
@@ -208,6 +217,11 @@ The :mod:`!binascii` module defines the following functions:
    Optional *alphabet* must be a :class:`bytes` object of length 32 which
    specifies an alternative alphabet.
 
+   If *padded* is true, the last group of 8 base 32 alphabet characters must
+   be padded with the '=' character.
+   If *padded* is false, the '=' character is treated as other non-alphabet
+   characters (depending on the value of *ignorechars*).
+
    *ignorechars* should be a :term:`bytes-like object` containing characters
    to ignore from the input.
    If *ignorechars* contains the pad character ``'='``,  the pad characters
@@ -218,14 +232,18 @@ The :mod:`!binascii` module defines the following functions:
 
    .. versionadded:: next
 
-.. function:: b2a_base32(data, /, *, alphabet=BASE32_ALPHABET, wrapcol=0)
+.. function:: b2a_base32(data, /, *, padded=True, alphabet=BASE32_ALPHABET, wrapcol=0)
 
    Convert binary data to a line of ASCII characters in base32 coding,
    as specified in :rfc:`4648`. The return value is the converted line.
 
    Optional *alphabet* must be a :term:`bytes-like object` of length 32 which
    specifies an alternative alphabet.
 
+   If *padded* is true (default), pad the encoded data with the '='
+   character to a size multiple of 8.
+   If *padded* is false, do not add the pad characters.
+
    If *wrapcol* is non-zero, insert a newline (``b'\n'``) character
    after at most every *wrapcol* characters.
    If *wrapcol* is zero (default), do not insert any newlines.

Doc/whatsnew/3.15.rst

@@ -657,6 +657,13 @@ base64
 * Added the *pad* parameter in :func:`~base64.z85encode`.
   (Contributed by Hauke Dämpfling in :gh:`143103`.)
 
+* Added the *padded* parameter in
+  :func:`~base64.b32encode`, :func:`~base64.b32decode`,
+  :func:`~base64.b32hexencode`, :func:`~base64.b32hexdecode`,
+  :func:`~base64.b64encode`, :func:`~base64.b64decode`,
+  :func:`~base64.urlsafe_b64encode`, and :func:`~base64.urlsafe_b64decode`.
+  (Contributed by Serhiy Storchaka in :gh:`73613`.)
+
 * Added the *wrapcol* parameter in :func:`~base64.b16encode`,
   :func:`~base64.b32encode`, :func:`~base64.b32hexencode`,
   :func:`~base64.b64encode`, :func:`~base64.b85encode`, and
@@ -686,6 +693,11 @@ binascii
 
   (Contributed by James Seo and Serhiy Storchaka in :gh:`101178`.)
 
+* Added the *padded* parameter in
+  :func:`~binascii.b2a_base32`, :func:`~binascii.a2b_base32`,
+  :func:`~binascii.b2a_base64`, and :func:`~binascii.a2b_base64`.
+  (Contributed by Serhiy Storchaka in :gh:`73613`.)
+
 * Added the *wrapcol* parameter in :func:`~binascii.b2a_base64`.
   (Contributed by Serhiy Storchaka in :gh:`143214`.)
 
@@ -2022,3 +2034,9 @@ that may require changes to your code.
   *dest* is now ``'foo'`` instead of ``'f'``.
   Pass an explicit *dest* argument to preserve the old behavior.
   (Contributed by Serhiy Storchaka in :gh:`138697`.)
+
+* Padding of input no longer required in :func:`base64.urlsafe_b64decode`.
+  Pass a new argument ``padded=True`` or use :func:`base64.b64decode`
+  with argument ``altchars=b'-_'`` (this works with older Python versions)
+  to make padding required.
+  (Contributed by Serhiy Storchaka in :gh:`73613`.)

Include/internal/pycore_global_strings.h

@@ -697,6 +697,7 @@ struct _Py_global_strings {
         STRUCT_FOR_ID(overlapped)
         STRUCT_FOR_ID(owner)
         STRUCT_FOR_ID(pad)
+        STRUCT_FOR_ID(padded)
         STRUCT_FOR_ID(pages)
         STRUCT_FOR_ID(parameter)
         STRUCT_FOR_ID(parent)