初始提交:terminalX 可运行态(M0/M1/M1.5 已真机验证)

- M0: libghostty SSH 终端(渲染/输入/连接)+ 白屏修复(OutputGate) + 会话状态机/自动重连
- M1: tsnet 用户态组网 + SSH-over-tsnet(fd 桥),shell 级真机验证;R5(Go+gvisor+C+++Swift 同进程) retire
- M1.5: tmux -CC 原生 tab(MVP)
- 结构: packages/(TXCore·TXTransport), apps/TerminalX, vendor/(libghostty-spm/libssh2/mbedtls/tsnet-bridge), artifacts/
- 文档: CLAUDE.md + docs/HANDOFF.md(新会话入口)
- 环境: 认证代理→依赖 vendor 本地化;Go 在 ~/.local/go;仅模拟器/未签名
- 待续: M2 mosh, tmux 多 pane, M4 安全(host key/SE)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
kid
2026-07-24 10:20:46 +08:00
commit aa92d0e676
2761 changed files with 803505 additions and 0 deletions

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,366 @@
# all-helpers.sh
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
# This file contains helpers for test components that are executed by all.sh.
# See "Files structure" in all-core.sh for other files used by all.sh.
#
# This file is the right place for helpers:
# - that are used by more than one component living in more than one file;
# - or (inclusive) that we want to share accross repos or branches.
#
# Helpers that are used in a single component file that is
# repo&branch-specific can be defined in the file where they are used.
################################################################
#### Helpers for components using libtestdriver1
################################################################
# How to use libtestdriver1
# -------------------------
#
# 1. Define the list algorithms and key types to accelerate,
# designated the same way as PSA_WANT_ macros but without PSA_WANT_.
# Examples:
# - loc_accel_list="ALG_JPAKE"
# - loc_accel_list="ALG_FFDH KEY_TYPE_DH_KEY_PAIR KEY_TYPE_DH_PUBLIC_KEY"
# 2. Make configurations changes for the driver and/or main libraries.
# 2a. Call helper_libtestdriver1_adjust_config <base>, where the argument
# can be either "default" to start with the default config, or a name
# supported by scripts/config.py (for example, "full"). This selects
# the base to use, and makes common adjustments.
# 2b. If desired, adjust the PSA_WANT symbols in psa/crypto_config.h.
# These changes affect both the driver and the main libraries.
# (Note: they need to have the same set of PSA_WANT symbols, as that
# determines the ABI between them.)
# 2c. Adjust MBEDTLS_ symbols in mbedtls_config.h. This only affects the
# main libraries. Typically, you want to disable the module(s) that are
# being accelerated. You may need to also disable modules that depend
# on them or options that are not supported with drivers.
# 2d. On top of psa/crypto_config.h, the driver library uses its own config
# file: tests/configs/config_test_driver.h. You usually don't need to
# edit it: using loc_extra_list (see below) is preferred. However, when
# there's no PSA symbol for what you want to enable, calling
# scripts/config.py on this file remains the only option.
# 3. Build the driver library, then the main libraries, test, and programs.
# 3a. Call helper_libtestdriver1_make_drivers "$loc_accel_list". You may
# need to enable more algorithms here, typically hash algorithms when
# accelerating some signature algorithms (ECDSA, RSAv2). This is done
# by passing a 2nd argument listing the extra algorithms.
# Example:
# loc_extra_list="ALG_SHA_224 ALG_SHA_256 ALG_SHA_384 ALG_SHA_512"
# helper_libtestdriver1_make_drivers "$loc_accel_list" "$loc_extra_list"
# 3b. Call helper_libtestdriver1_make_main "$loc_accel_list". Any
# additional arguments will be passed to make: this can be useful if
# you don't want to build everything when iterating during development.
# Example:
# helper_libtestdriver1_make_main "$loc_accel_list" -C tests test_suite_foo
# 4. Run the tests you want.
# Adjust the configuration - for both libtestdriver1 and main library,
# as they should have the same PSA_WANT macros.
helper_libtestdriver1_adjust_config() {
base_config=$1
# Select the base configuration
if [ "$base_config" != "default" ]; then
scripts/config.py "$base_config"
fi
if in_mbedtls_repo && in_3_6_branch; then
# Enable PSA-based config (necessary to use drivers)
# MBEDTLS_PSA_CRYPTO_CONFIG is a legacy setting which should only be set on 3.6 LTS branches.
scripts/config.py set MBEDTLS_PSA_CRYPTO_CONFIG
# Dynamic secure element support is a deprecated feature and needs to be disabled here.
# This is done to have the same form of psa_key_attributes_s for libdriver and library.
scripts/config.py unset MBEDTLS_PSA_CRYPTO_SE_C
fi
# If threading is enabled on the normal build, then we need to enable it in the drivers as well,
# otherwise we will end up running multithreaded tests without mutexes to protect them.
if scripts/config.py get MBEDTLS_THREADING_C; then
if in_3_6_branch; then
scripts/config.py -f "$CONFIG_TEST_DRIVER_H" set MBEDTLS_THREADING_C
else
scripts/config.py -c "$CONFIG_TEST_DRIVER_H" set MBEDTLS_THREADING_C
fi
fi
if scripts/config.py get MBEDTLS_THREADING_PTHREAD; then
if in_3_6_branch; then
scripts/config.py -f "$CONFIG_TEST_DRIVER_H" set MBEDTLS_THREADING_PTHREAD
else
scripts/config.py -c "$CONFIG_TEST_DRIVER_H" set MBEDTLS_THREADING_PTHREAD
fi
fi
}
# Build the drivers library libtestdriver1.a (with ASan).
#
# Parameters:
# 1. a space-separated list of things to accelerate;
# 2. optional: a space-separate list of things to also support.
# Here "things" are PSA_WANT_ symbols but with PSA_WANT_ removed.
helper_libtestdriver1_make_drivers() {
loc_accel_flags=$( echo "$1 ${2-}" | sed 's/[^ ]* */-DLIBTESTDRIVER1_MBEDTLS_PSA_ACCEL_&/g' )
make CC=$ASAN_CC -C tests libtestdriver1.a CFLAGS=" $ASAN_CFLAGS $loc_accel_flags" LDFLAGS="$ASAN_CFLAGS"
}
# Build the main libraries, programs and tests,
# linking to the drivers library (with ASan).
#
# Parameters:
# 1. a space-separated list of things to accelerate;
# *. remaining arguments if any are passed directly to make
# (examples: lib, -C tests test_suite_xxx, etc.)
# Here "things" are PSA_WANT_ symbols but with PSA_WANT_ removed.
helper_libtestdriver1_make_main() {
loc_accel_list=$1
shift
# we need flags both with and without the LIBTESTDRIVER1_ prefix
loc_accel_flags=$( echo "$loc_accel_list" | sed 's/[^ ]* */-DLIBTESTDRIVER1_MBEDTLS_PSA_ACCEL_&/g' )
loc_accel_flags="$loc_accel_flags $( echo "$loc_accel_list" | sed 's/[^ ]* */-DMBEDTLS_PSA_ACCEL_&/g' )"
$MAKE_COMMAND CC=$ASAN_CC CFLAGS="$ASAN_CFLAGS -I../tests/include -I../framework/tests/include -I../tests -I../../tests -DPSA_CRYPTO_DRIVER_TEST -DMBEDTLS_TEST_LIBTESTDRIVER1 $loc_accel_flags" LDFLAGS="-ltestdriver1 $ASAN_CFLAGS" "$@"
}
################################################################
#### Helpers for components using psasim
################################################################
# Set some default values $CONFIG_H in order to build server or client sides
# in PSASIM. There is only 1 mandatory parameter:
# - $1: target which can be "client" or "server"
helper_psasim_config() {
TARGET=$1
if [ "$TARGET" == "client" ]; then
scripts/config.py full
scripts/config.py unset MBEDTLS_PSA_CRYPTO_C
scripts/config.py unset MBEDTLS_PSA_CRYPTO_STORAGE_C
scripts/config.py unset MBEDTLS_ENTROPY_NV_SEED
scripts/config.py unset MBEDTLS_PLATFORM_NV_SEED_ALT
if in_mbedtls_repo && in_3_6_branch; then
# Dynamic secure element support is a deprecated feature and it is not
# available when CRYPTO_C and PSA_CRYPTO_STORAGE_C are disabled.
scripts/config.py unset MBEDTLS_PSA_CRYPTO_SE_C
fi
# Disable potentially problematic features
scripts/config.py unset MBEDTLS_X509_RSASSA_PSS_SUPPORT
scripts/config.py unset MBEDTLS_KEY_EXCHANGE_ECDHE_RSA_ENABLED
scripts/config.py unset MBEDTLS_ECP_RESTARTABLE
scripts/config.py unset MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER
scripts/config.py unset MBEDTLS_PK_PARSE_EC_EXTENDED
scripts/config.py unset MBEDTLS_PK_PARSE_EC_COMPRESSED
scripts/config.py unset-all MBEDTLS_SHA256_USE_.*_CRYPTO_
scripts/config.py unset-all MBEDTLS_SHA512_USE_.*_CRYPTO_
else
scripts/config.py crypto_full
scripts/config.py unset MBEDTLS_PSA_CRYPTO_BUILTIN_KEYS
if in_mbedtls_repo && in_3_6_branch; then
# We need to match the client with MBEDTLS_PSA_CRYPTO_SE_C
scripts/config.py unset MBEDTLS_PSA_CRYPTO_SE_C
fi
# Also ensure MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER not set (to match client)
scripts/config.py unset MBEDTLS_PSA_CRYPTO_KEY_ID_ENCODES_OWNER
fi
}
# This is a helper function to be used in psasim builds. It is meant to clean
# up the library's workspace after the server build and before the client
# build. Built libraries (mbedcrypto, mbedx509 and mbedtls) are supposed to be
# already copied to psasim folder at this point.
helper_psasim_cleanup_before_client() {
# Clean up library files
make -C library clean
# Restore files that were backup before building library files. This
# includes $CONFIG_H and $CRYPTO_CONFIG_H.
restore_backed_up_files
}
# Helper to build the libraries for client/server in PSASIM. If the server is
# being built, then it builds also the final executable.
# There is only 1 mandatory parameter:
# - $1: target which can be "client" or "server"
helper_psasim_build() {
TARGET=$1
shift
TARGET_LIB=${TARGET}_libs
make -C $PSASIM_PATH CFLAGS="$ASAN_CFLAGS" LDFLAGS="$ASAN_CFLAGS" $TARGET_LIB "$@"
# Build also the server application after its libraries have been built.
if [ "$TARGET" == "server" ]; then
make -C $PSASIM_PATH CFLAGS="$ASAN_CFLAGS" LDFLAGS="$ASAN_CFLAGS" test/psa_server
fi
}
################################################################
#### Configuration helpers
################################################################
# When called with no parameter this function disables all builtin curves.
# The function optionally accepts 1 parameter: a space-separated list of the
# curves that should be kept enabled.
helper_disable_builtin_curves() {
allowed_list="${1:-}"
scripts/config.py unset-all "MBEDTLS_ECP_DP_[0-9A-Z_a-z]*_ENABLED"
for curve in $allowed_list; do
scripts/config.py set $curve
done
}
# Helper returning the list of supported elliptic curves from CRYPTO_CONFIG_H,
# without the "PSA_WANT_" prefix. This becomes handy for accelerating curves
# in the following helpers.
helper_get_psa_curve_list () {
loc_list=""
for item in $(sed -n 's/^#define PSA_WANT_\(ECC_[0-9A-Z_a-z]*\).*/\1/p' <"$CRYPTO_CONFIG_H"); do
loc_list="$loc_list $item"
done
echo "$loc_list"
}
# Helper returning the list of supported DH groups from CRYPTO_CONFIG_H,
# without the "PSA_WANT_" prefix. This becomes handy for accelerating DH groups
# in the following helpers.
helper_get_psa_dh_group_list () {
loc_list=""
for item in $(sed -n 's/^#define PSA_WANT_\(DH_RFC7919_[0-9]*\).*/\1/p' <"$CRYPTO_CONFIG_H"); do
loc_list="$loc_list $item"
done
echo "$loc_list"
}
# Get the list of uncommented PSA_WANT_KEY_TYPE_xxx_ from CRYPTO_CONFIG_H. This
# is useful to easily get a list of key type symbols to accelerate.
# The function accepts a single argument which is the key type: ECC, DH, RSA.
helper_get_psa_key_type_list() {
key_type="$1"
loc_list=""
for item in $(sed -n "s/^#define PSA_WANT_\(KEY_TYPE_${key_type}_[0-9A-Z_a-z]*\).*/\1/p" <"$CRYPTO_CONFIG_H"); do
# Skip DERIVE for elliptic keys since there is no driver dispatch for
# it so it cannot be accelerated.
if [ "$item" != "KEY_TYPE_ECC_KEY_PAIR_DERIVE" ]; then
loc_list="$loc_list $item"
fi
done
echo "$loc_list"
}
################################################################
#### Misc. helpers for components
################################################################
helper_armc6_build_test()
{
FLAGS="$1"
msg "build: ARM Compiler 6 ($FLAGS)"
$MAKE_COMMAND clean
ARM_TOOL_VARIANT="ult" CC="$ARMC6_CC" AR="$ARMC6_AR" CFLAGS="$FLAGS" \
WARNING_CFLAGS='-Werror -xc -std=c99' $MAKE_COMMAND lib
msg "size: ARM Compiler 6 ($FLAGS)"
"$ARMC6_FROMELF" -z library/*.o
if [ -n "${PSA_CORE_PATH}" ]; then
"$ARMC6_FROMELF" -z ${PSA_CORE_PATH}/*.o
fi
if [ -n "${BUILTIN_SRC_PATH}" ]; then
"$ARMC6_FROMELF" -z ${BUILTIN_SRC_PATH}/*.o
fi
}
helper_armc6_cmake_build_test()
{
FLAGS="$1"
msg "build: CMake + ARM Compiler 6 ($FLAGS)"
cmake -DCMAKE_SYSTEM_NAME="Generic" -DCMAKE_SYSTEM_PROCESSOR="cortex-m0" \
-DCMAKE_C_COMPILER="$ARMC6_CC" -DCMAKE_C_LINKER="$ARMC6_LINK" \
-DCMAKE_AR="$ARMC6_AR" -DCMAKE_C_FLAGS="$FLAGS" \
-DCMAKE_C_COMPILER_WORKS=TRUE -DENABLE_TESTING=OFF \
-DENABLE_PROGRAMS=OFF "$TF_PSA_CRYPTO_ROOT_DIR"
make
msg "size: ARM Compiler 6 ($FLAGS)"
"$ARMC6_FROMELF" -z ${PSA_CORE_PATH}/CMakeFiles/tfpsacrypto.dir/*.o
"$ARMC6_FROMELF" -z ${BUILTIN_SRC_PATH}/../CMakeFiles/builtin.dir/src/*.o
}
clang_version() {
if command -v clang > /dev/null ; then
clang --version|grep version|sed -E 's#.*version ([0-9]+).*#\1#'
else
echo 0 # report version 0 for "no clang"
fi
}
gcc_version() {
gcc="$1"
if command -v "$gcc" > /dev/null ; then
"$gcc" --version | sed -En '1s/^[^ ]* \([^)]*\) ([0-9]+).*/\1/p'
else
echo 0 # report version 0 for "no gcc"
fi
}
can_run_cc_output() {
cc="$1"
result=false
if type "$cc" >/dev/null 2>&1; then
testbin=$(mktemp)
if echo 'int main(void){return 0;}' | "$cc" -o "$testbin" -x c -; then
if "$testbin" 2>/dev/null; then
result=true
fi
fi
rm -f "$testbin"
fi
$result
}
can_run_arm_linux_gnueabi=
can_run_arm_linux_gnueabi () {
if [ -z "$can_run_arm_linux_gnueabi" ]; then
if can_run_cc_output "${ARM_LINUX_GNUEABI_GCC_PREFIX}gcc"; then
can_run_arm_linux_gnueabi=true
else
can_run_arm_linux_gnueabi=false
fi
fi
$can_run_arm_linux_gnueabi
}
can_run_arm_linux_gnueabihf=
can_run_arm_linux_gnueabihf () {
if [ -z "$can_run_arm_linux_gnueabihf" ]; then
if can_run_cc_output "${ARM_LINUX_GNUEABIHF_GCC_PREFIX}gcc"; then
can_run_arm_linux_gnueabihf=true
else
can_run_arm_linux_gnueabihf=false
fi
fi
$can_run_arm_linux_gnueabihf
}
can_run_aarch64_linux_gnu=
can_run_aarch64_linux_gnu () {
if [ -z "$can_run_aarch64_linux_gnu" ]; then
if can_run_cc_output "${AARCH64_LINUX_GNU_GCC_PREFIX}gcc"; then
can_run_aarch64_linux_gnu=true
else
can_run_aarch64_linux_gnu=false
fi
fi
$can_run_aarch64_linux_gnu
}

View File

@@ -0,0 +1,69 @@
#!/bin/sh
# Generate doxygen documentation with a full mbedtls_config.h (this ensures that every
# available flag is documented, and avoids warnings about documentation
# without a corresponding #define).
#
# /!\ This must not be a Makefile target, as it would create a race condition
# when multiple targets are invoked in the same parallel build.
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
set -eu
. $(dirname "$0")/project_detection.sh
if in_mbedtls_repo; then
CONFIG_H='include/mbedtls/mbedtls_config.h'
if [ -r $CONFIG_H ]; then :; else
echo "$CONFIG_H not found" >&2
fi
if ! in_3_6_branch; then
CRYPTO_CONFIG_H='tf-psa-crypto/include/psa/crypto_config.h'
fi
CONFIG_BAK=${CONFIG_H}.bak
cp -p $CONFIG_H $CONFIG_BAK
fi
if in_tf_psa_crypto_repo; then
CRYPTO_CONFIG_H='include/psa/crypto_config.h'
fi
if in_tf_psa_crypto_repo || (in_mbedtls_repo && ! in_3_6_branch); then
if [ -r $CRYPTO_CONFIG_H ]; then :; else
echo "$CRYPTO_CONFIG_H not found" >&2
exit 1
fi
CRYPTO_CONFIG_BAK=${CRYPTO_CONFIG_H}.bak
cp -p $CRYPTO_CONFIG_H $CRYPTO_CONFIG_BAK
fi
if in_mbedtls_repo && in_3_6_branch; then
scripts/config.py realfull
make apidoc
else
scripts/config.py realfull
ROOT_DIR=$PWD
rm -rf doxygen/build-apidoc-full
mkdir doxygen/build-apidoc-full
cd doxygen/build-apidoc-full
cmake -DCMAKE_BUILD_TYPE:String=Check -DGEN_FILES=ON $ROOT_DIR
if in_mbedtls_repo; then
cmake --build . --target mbedtls-apidoc
else
cmake --build . --target tfpsacrypto-apidoc
fi
cd $ROOT_DIR
# The documentation is built in the source tree thus we can delete the
# build tree.
rm -rf doxygen/build-apidoc-full
fi
if in_mbedtls_repo; then
mv $CONFIG_BAK $CONFIG_H
fi
if in_tf_psa_crypto_repo || (in_mbedtls_repo && ! in_3_6_branch); then
mv $CRYPTO_CONFIG_BAK $CRYPTO_CONFIG_H
fi

View File

@@ -0,0 +1,525 @@
#!/usr/bin/env python3
"""Assemble Mbed TLS change log entries into the change log file.
Add changelog entries to the first level-2 section.
Create a new level-2 section for unreleased changes if needed.
Remove the input files unless --keep-entries is specified.
In each level-3 section, entries are sorted in chronological order
(oldest first). From oldest to newest:
* Merged entry files are sorted according to their merge date (date of
the merge commit that brought the commit that created the file into
the target branch).
* Committed but unmerged entry files are sorted according to the date
of the commit that adds them.
* Uncommitted entry files are sorted according to their modification time.
You must run this program from within a git working directory.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
from collections import OrderedDict, namedtuple
import datetime
import functools
import glob
import os
import re
import subprocess
import sys
class InputFormatError(Exception):
def __init__(self, filename, line_number, message, *args, **kwargs):
message = '{}:{}: {}'.format(filename, line_number,
message.format(*args, **kwargs))
super().__init__(message)
class CategoryParseError(Exception):
def __init__(self, line_offset, error_message):
self.line_offset = line_offset
self.error_message = error_message
super().__init__('{}: {}'.format(line_offset, error_message))
class LostContent(Exception):
def __init__(self, filename, line):
message = ('Lost content from {}: "{}"'.format(filename, line))
super().__init__(message)
class FilePathError(Exception):
def __init__(self, filenames):
message = ('Changelog filenames do not end with .txt: {}'.format(", ".join(filenames)))
super().__init__(message)
# The category names we use in the changelog.
# If you edit this, update ChangeLog.d/README.md.
STANDARD_CATEGORIES = (
'API changes',
'Default behavior changes',
'Requirement changes',
'New deprecations',
'Removals',
'Features',
'Security',
'Bugfix',
'Changes',
)
# The maximum line length for an entry
MAX_LINE_LENGTH = 80
CategoryContent = namedtuple('CategoryContent', [
'name', 'title_line', # Title text and line number of the title
'body', 'body_line', # Body text and starting line number of the body
])
class ChangelogFormat:
"""Virtual class documenting how to write a changelog format class."""
@classmethod
def extract_top_version(cls, changelog_file_content):
"""Split out the top version section.
If the top version is already released, create a new top
version section for an unreleased version.
Return ``(header, top_version_title, top_version_body, trailer)``
where the "top version" is the existing top version section if it's
for unreleased changes, and a newly created section otherwise.
To assemble the changelog after modifying top_version_body,
concatenate the four pieces.
"""
raise NotImplementedError
@classmethod
def split_categories(cls, version_body):
"""Split a changelog version section body into categories.
Return a list of `CategoryContent` the name is category title
without any formatting.
"""
raise NotImplementedError
@classmethod
def format_category(cls, title, body):
"""Construct the text of a category section from its title and body."""
raise NotImplementedError
class TextChangelogFormat(ChangelogFormat):
"""The traditional Mbed TLS changelog format."""
_unreleased_version_text = '= {} x.x.x branch released xxxx-xx-xx'
@classmethod
def is_released_version(cls, title):
# Look for an incomplete release date
return not re.search(r'[0-9x]{4}-[0-9x]{2}-[0-9x]?x', title)
_top_version_re = re.compile(r'(?:\A|\n)(=[^\n]*\n+)(.*?\n)(?:=|$)',
re.DOTALL)
_name_re = re.compile(r'=\s(.*)\s[0-9x]+\.', re.DOTALL)
@classmethod
def extract_top_version(cls, changelog_file_content):
"""A version section starts with a line starting with '='."""
m = re.search(cls._top_version_re, changelog_file_content)
top_version_start = m.start(1)
top_version_end = m.end(2)
top_version_title = m.group(1)
top_version_body = m.group(2)
name = re.match(cls._name_re, top_version_title).group(1)
if cls.is_released_version(top_version_title):
top_version_end = top_version_start
top_version_title = cls._unreleased_version_text.format(name) + '\n\n'
top_version_body = ''
return (changelog_file_content[:top_version_start],
top_version_title, top_version_body,
changelog_file_content[top_version_end:])
_category_title_re = re.compile(r'(^\w.*)\n+', re.MULTILINE)
@classmethod
def split_categories(cls, version_body):
"""A category title is a line with the title in column 0."""
if not version_body:
return []
title_matches = list(re.finditer(cls._category_title_re, version_body))
if not title_matches or title_matches[0].start() != 0:
# There is junk before the first category.
raise CategoryParseError(0, 'Junk found where category expected')
title_starts = [m.start(1) for m in title_matches]
body_starts = [m.end(0) for m in title_matches]
body_ends = title_starts[1:] + [len(version_body)]
bodies = [version_body[body_start:body_end].rstrip('\n') + '\n'
for (body_start, body_end) in zip(body_starts, body_ends)]
title_lines = [version_body[:pos].count('\n') for pos in title_starts]
body_lines = [version_body[:pos].count('\n') for pos in body_starts]
return [CategoryContent(title_match.group(1), title_line,
body, body_line)
for title_match, title_line, body, body_line
in zip(title_matches, title_lines, bodies, body_lines)]
@classmethod
def format_category(cls, title, body):
# `split_categories` ensures that each body ends with a newline.
# Make sure that there is additionally a blank line between categories.
if not body.endswith('\n\n'):
body += '\n'
return title + '\n' + body
class ChangeLog:
"""An Mbed TLS changelog.
A changelog file consists of some header text followed by one or
more version sections. The version sections are in reverse
chronological order. Each version section consists of a title and a body.
The body of a version section consists of zero or more category
subsections. Each category subsection consists of a title and a body.
A changelog entry file has the same format as the body of a version section.
A `ChangelogFormat` object defines the concrete syntax of the changelog.
Entry files must have the same format as the changelog file.
"""
# Only accept dotted version numbers (e.g. "3.1", not "3").
# Refuse ".x" in a version number where x is a letter: this indicates
# a version that is not yet released. Something like "3.1a" is accepted.
_version_number_re = re.compile(r'[0-9]+\.[0-9A-Za-z.]+')
_incomplete_version_number_re = re.compile(r'.*\.[A-Za-z]')
_only_url_re = re.compile(r'^\s*\w+://\S+\s*$')
_has_url_re = re.compile(r'.*://.*')
def add_categories_from_text(self, filename, line_offset,
text, allow_unknown_category):
"""Parse a version section or entry file."""
try:
categories = self.format.split_categories(text)
except CategoryParseError as e:
raise InputFormatError(filename, line_offset + e.line_offset,
e.error_message)
for category in categories:
if not allow_unknown_category and \
category.name not in self.categories:
raise InputFormatError(filename,
line_offset + category.title_line,
'Unknown category: "{}"',
category.name)
body_split = category.body.splitlines()
for line_number, line in enumerate(body_split, 1):
if not self._only_url_re.match(line) and \
len(line) > MAX_LINE_LENGTH:
long_url_msg = '. URL exceeding length limit must be alone in its line.' \
if self._has_url_re.match(line) else ""
raise InputFormatError(filename,
category.body_line + line_number,
'Line is longer than allowed: '
'Length {} (Max {}){}',
len(line), MAX_LINE_LENGTH,
long_url_msg)
self.categories[category.name] += category.body
def __init__(self, input_stream, changelog_format):
"""Create a changelog object.
Populate the changelog object from the content of the file
input_stream.
"""
self.format = changelog_format
whole_file = input_stream.read()
(self.header,
self.top_version_title, top_version_body,
self.trailer) = self.format.extract_top_version(whole_file)
# Split the top version section into categories.
self.categories = OrderedDict()
for category in STANDARD_CATEGORIES:
self.categories[category] = ''
offset = (self.header + self.top_version_title).count('\n') + 1
self.add_categories_from_text(input_stream.name, offset,
top_version_body, True)
def add_file(self, input_stream):
"""Add changelog entries from a file.
"""
self.add_categories_from_text(input_stream.name, 1,
input_stream.read(), False)
def write(self, filename):
"""Write the changelog to the specified file.
"""
with open(filename, 'w', encoding='utf-8') as out:
out.write(self.header)
out.write(self.top_version_title)
for title, body in self.categories.items():
if not body:
continue
out.write(self.format.format_category(title, body))
out.write(self.trailer)
@functools.total_ordering
class EntryFileSortKey:
"""This classes defines an ordering on changelog entry files: older < newer.
* Merged entry files are sorted according to their merge date (date of
the merge commit that brought the commit that created the file into
the target branch).
* Committed but unmerged entry files are sorted according to the date
of the commit that adds them.
* Uncommitted entry files are sorted according to their modification time.
This class assumes that the file is in a git working directory with
the target branch checked out.
"""
# Categories of files. A lower number is considered older.
MERGED = 0
COMMITTED = 1
LOCAL = 2
@staticmethod
def creation_hash(filename):
"""Return the git commit id at which the given file was created.
Return None if the file was never checked into git.
"""
hashes = subprocess.check_output(['git', 'log', '--format=%H',
'--follow',
'--', filename])
m = re.search('(.+)$', hashes.decode('ascii'))
if not m:
# The git output is empty. This means that the file was
# never checked in.
return None
# The last commit in the log is the oldest one, which is when the
# file was created.
return m.group(0)
@staticmethod
def list_merges(some_hash, target, *options):
"""List merge commits from some_hash to target.
Pass options to git to select which commits are included.
"""
text = subprocess.check_output(['git', 'rev-list',
'--merges', *options,
'..'.join([some_hash, target])])
return text.decode('ascii').rstrip('\n').split('\n')
@classmethod
def merge_hash(cls, some_hash):
"""Return the git commit id at which the given commit was merged.
Return None if the given commit was never merged.
"""
target = 'HEAD'
# List the merges from some_hash to the target in two ways.
# The ancestry list is the ones that are both descendants of
# some_hash and ancestors of the target.
ancestry = frozenset(cls.list_merges(some_hash, target,
'--ancestry-path'))
# The first_parents list only contains merges that are directly
# on the target branch. We want it in reverse order (oldest first).
first_parents = cls.list_merges(some_hash, target,
'--first-parent', '--reverse')
# Look for the oldest merge commit that's both on the direct path
# and directly on the target branch. That's the place where some_hash
# was merged on the target branch. See
# https://stackoverflow.com/questions/8475448/find-merge-commit-which-include-a-specific-commit
for commit in first_parents:
if commit in ancestry:
return commit
return None
@staticmethod
def commit_timestamp(commit_id):
"""Return the timestamp of the given commit."""
text = subprocess.check_output(['git', 'show', '-s',
'--format=%ct',
commit_id])
return datetime.datetime.utcfromtimestamp(int(text))
@staticmethod
def file_timestamp(filename):
"""Return the modification timestamp of the given file."""
mtime = os.stat(filename).st_mtime
return datetime.datetime.fromtimestamp(mtime)
def __init__(self, filename):
"""Determine position of the file in the changelog entry order.
This constructor returns an object that can be used with comparison
operators, with `sort` and `sorted`, etc. Older entries are sorted
before newer entries.
"""
self.filename = filename
creation_hash = self.creation_hash(filename)
if not creation_hash:
self.category = self.LOCAL
self.datetime = self.file_timestamp(filename)
return
merge_hash = self.merge_hash(creation_hash)
if not merge_hash:
self.category = self.COMMITTED
self.datetime = self.commit_timestamp(creation_hash)
return
self.category = self.MERGED
self.datetime = self.commit_timestamp(merge_hash)
def sort_key(self):
""""Return a concrete sort key for this entry file sort key object.
``ts1 < ts2`` is implemented as ``ts1.sort_key() < ts2.sort_key()``.
"""
return (self.category, self.datetime, self.filename)
def __eq__(self, other):
return self.sort_key() == other.sort_key()
def __lt__(self, other):
return self.sort_key() < other.sort_key()
def check_output(generated_output_file, main_input_file, merged_files):
"""Make sanity checks on the generated output.
The intent of these sanity checks is to have reasonable confidence
that no content has been lost.
The sanity check is that every line that is present in an input file
is also present in an output file. This is not perfect but good enough
for now.
"""
with open(generated_output_file, 'r', encoding='utf-8') as fd:
generated_output = set(fd)
for line in open(main_input_file, 'r', encoding='utf-8'):
if line not in generated_output:
raise LostContent('original file', line)
for merged_file in merged_files:
for line in open(merged_file, 'r', encoding='utf-8'):
if line not in generated_output:
raise LostContent(merged_file, line)
def finish_output(changelog, output_file, input_file, merged_files):
"""Write the changelog to the output file.
The input file and the list of merged files are used only for sanity
checks on the output.
"""
if os.path.exists(output_file) and not os.path.isfile(output_file):
# The output is a non-regular file (e.g. pipe). Write to it directly.
output_temp = output_file
else:
# The output is a regular file. Write to a temporary file,
# then move it into place atomically.
output_temp = output_file + '.tmp'
changelog.write(output_temp)
check_output(output_temp, input_file, merged_files)
if output_temp != output_file:
os.rename(output_temp, output_file)
def remove_merged_entries(files_to_remove):
for filename in files_to_remove:
os.remove(filename)
def list_files_to_merge(options):
"""List the entry files to merge, oldest first.
"Oldest" is defined by `EntryFileSortKey`.
Also check for required .txt extension
"""
files_to_merge = glob.glob(os.path.join(options.dir, '*'))
# Ignore 00README.md
readme = os.path.join(options.dir, "00README.md")
if readme in files_to_merge:
files_to_merge.remove(readme)
# Identify files without the required .txt extension
bad_files = [x for x in files_to_merge if not x.endswith(".txt")]
if bad_files:
raise FilePathError(bad_files)
files_to_merge.sort(key=EntryFileSortKey)
return files_to_merge
def merge_entries(options):
"""Merge changelog entries into the changelog file.
Read the changelog file from options.input.
Check that all entries have a .txt extension
Read entries to merge from the directory options.dir.
Write the new changelog to options.output.
Remove the merged entries if options.keep_entries is false.
"""
with open(options.input, 'r', encoding='utf-8') as input_file:
changelog = ChangeLog(input_file, TextChangelogFormat)
files_to_merge = list_files_to_merge(options)
if not files_to_merge:
sys.stderr.write('There are no pending changelog entries.\n')
return
for filename in files_to_merge:
with open(filename, 'r', encoding='utf-8') as input_file:
changelog.add_file(input_file)
finish_output(changelog, options.output, options.input, files_to_merge)
if not options.keep_entries:
remove_merged_entries(files_to_merge)
def show_file_timestamps(options):
"""List the files to merge and their timestamp.
This is only intended for debugging purposes.
"""
files = list_files_to_merge(options)
for filename in files:
ts = EntryFileSortKey(filename)
print(ts.category, ts.datetime, filename)
def set_defaults(options):
"""Add default values for missing options."""
output_file = getattr(options, 'output', None)
if output_file is None:
options.output = options.input
if getattr(options, 'keep_entries', None) is None:
options.keep_entries = (output_file is not None)
def main():
"""Command line entry point."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--dir', '-d', metavar='DIR',
default='ChangeLog.d',
help='Directory to read entries from'
' (default: ChangeLog.d)')
parser.add_argument('--input', '-i', metavar='FILE',
default='ChangeLog',
help='Existing changelog file to read from and augment'
' (default: ChangeLog)')
parser.add_argument('--keep-entries',
action='store_true', dest='keep_entries', default=None,
help='Keep the files containing entries'
' (default: remove them if --output/-o is not specified)')
parser.add_argument('--no-keep-entries',
action='store_false', dest='keep_entries',
help='Remove the files containing entries after they are merged'
' (default: remove them if --output/-o is not specified)')
parser.add_argument('--output', '-o', metavar='FILE',
help='Output changelog file'
' (default: overwrite the input)')
parser.add_argument('--list-files-only',
action='store_true',
help=('Only list the files that would be processed '
'(with some debugging information)'))
options = parser.parse_args()
set_defaults(options)
if options.list_files_only:
show_file_timestamps(options)
return
merge_entries(options)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,468 @@
#!/usr/bin/env python3
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
"""Audit validity date of X509 crt/crl/csr.
This script is used to audit the validity date of crt/crl/csr used for testing.
It prints the information about X.509 objects excluding the objects that
are valid throughout the desired validity period. The data are collected
from framework/data_files/ and tests/suites/*.data files by default.
"""
import os
import re
import typing
import argparse
import datetime
import glob
import logging
import hashlib
from enum import Enum
# The script requires cryptography >= 35.0.0 which is only available
# for Python >= 3.6.
import cryptography
from cryptography import x509
from generate_test_code import FileWrapper
from mbedtls_framework import build_tree
from mbedtls_framework import logging_util
def check_cryptography_version():
match = re.match(r'^[0-9]+', cryptography.__version__)
if match is None or int(match.group(0)) < 35:
raise Exception("audit-validity-dates requires cryptography >= 35.0.0"
+ "({} is too old)".format(cryptography.__version__))
class DataType(Enum):
CRT = 1 # Certificate
CRL = 2 # Certificate Revocation List
CSR = 3 # Certificate Signing Request
class DataFormat(Enum):
PEM = 1 # Privacy-Enhanced Mail
DER = 2 # Distinguished Encoding Rules
class AuditData:
"""Store data location, type and validity period of X.509 objects."""
#pylint: disable=too-few-public-methods
def __init__(self, data_type: DataType, x509_obj):
self.data_type = data_type
# the locations that the x509 object could be found
self.locations = [] # type: typing.List[str]
self.fill_validity_duration(x509_obj)
self._obj = x509_obj
encoding = cryptography.hazmat.primitives.serialization.Encoding.DER
self._identifier = hashlib.sha1(self._obj.public_bytes(encoding)).hexdigest()
@property
def identifier(self):
"""
Identifier of the underlying X.509 object, which is consistent across
different runs.
"""
return self._identifier
def fill_validity_duration(self, x509_obj):
"""Read validity period from an X.509 object."""
# Certificate expires after "not_valid_after"
# Certificate is invalid before "not_valid_before"
if self.data_type == DataType.CRT:
self.not_valid_after = x509_obj.not_valid_after
self.not_valid_before = x509_obj.not_valid_before
# CertificateRevocationList expires after "next_update"
# CertificateRevocationList is invalid before "last_update"
elif self.data_type == DataType.CRL:
self.not_valid_after = x509_obj.next_update
self.not_valid_before = x509_obj.last_update
# CertificateSigningRequest is always valid.
elif self.data_type == DataType.CSR:
self.not_valid_after = datetime.datetime.max
self.not_valid_before = datetime.datetime.min
else:
raise ValueError("Unsupported file_type: {}".format(self.data_type))
class X509Parser:
"""A parser class to parse crt/crl/csr file or data in PEM/DER format."""
PEM_REGEX = br'-{5}BEGIN (?P<type>.*?)-{5}(?P<data>.*?)-{5}END (?P=type)-{5}'
PEM_TAG_REGEX = br'-{5}BEGIN (?P<type>.*?)-{5}\n'
PEM_TAGS = {
DataType.CRT: 'CERTIFICATE',
DataType.CRL: 'X509 CRL',
DataType.CSR: 'CERTIFICATE REQUEST'
}
def __init__(self,
backends:
typing.Dict[DataType,
typing.Dict[DataFormat,
typing.Callable[[bytes], object]]]) \
-> None:
self.backends = backends
self.__generate_parsers()
def __generate_parser(self, data_type: DataType):
"""Parser generator for a specific DataType"""
tag = self.PEM_TAGS[data_type]
pem_loader = self.backends[data_type][DataFormat.PEM]
der_loader = self.backends[data_type][DataFormat.DER]
def wrapper(data: bytes):
pem_type = X509Parser.pem_data_type(data)
# It is in PEM format with target tag
if pem_type == tag:
return pem_loader(data)
# It is in PEM format without target tag
if pem_type:
return None
# It might be in DER format
try:
result = der_loader(data)
except ValueError:
result = None
return result
wrapper.__name__ = "{}.parser[{}]".format(type(self).__name__, tag)
return wrapper
def __generate_parsers(self):
"""Generate parsers for all support DataType"""
self.parsers = {}
for data_type, _ in self.PEM_TAGS.items():
self.parsers[data_type] = self.__generate_parser(data_type)
def __getitem__(self, item):
return self.parsers[item]
@staticmethod
def pem_data_type(data: bytes) -> typing.Optional[str]:
"""Get the tag from the data in PEM format
:param data: data to be checked in binary mode.
:return: PEM tag or "" when no tag detected.
"""
m = re.search(X509Parser.PEM_TAG_REGEX, data)
if m is not None:
return m.group('type').decode('UTF-8')
else:
return None
@staticmethod
def check_hex_string(hex_str: str) -> bool:
"""Check if the hex string is possibly DER data."""
hex_len = len(hex_str)
# At least 6 hex char for 3 bytes: Type + Length + Content
if hex_len < 6:
return False
# Check if Type (1 byte) is SEQUENCE.
if hex_str[0:2] != '30':
return False
# Check LENGTH (1 byte) value
content_len = int(hex_str[2:4], base=16)
consumed = 4
if content_len in (128, 255):
# Indefinite or Reserved
return False
elif content_len > 127:
# Definite, Long
length_len = (content_len - 128) * 2
content_len = int(hex_str[consumed:consumed+length_len], base=16)
consumed += length_len
# Check LENGTH
if hex_len != content_len * 2 + consumed:
return False
return True
class Auditor:
"""
A base class that uses X509Parser to parse files to a list of AuditData.
A subclass must implement the following methods:
- collect_default_files: Return a list of file names that are defaultly
used for parsing (auditing). The list will be stored in
Auditor.default_files.
- parse_file: Method that parses a single file to a list of AuditData.
A subclass may override the following methods:
- parse_bytes: Defaultly, it parses `bytes` that contains only one valid
X.509 data(DER/PEM format) to an X.509 object.
- walk_all: Defaultly, it iterates over all the files in the provided
file name list, calls `parse_file` for each file and stores the results
by extending the `results` passed to the function.
"""
def __init__(self, logger):
self.logger = logger
self.default_files = self.collect_default_files()
self.parser = X509Parser({
DataType.CRT: {
DataFormat.PEM: x509.load_pem_x509_certificate,
DataFormat.DER: x509.load_der_x509_certificate
},
DataType.CRL: {
DataFormat.PEM: x509.load_pem_x509_crl,
DataFormat.DER: x509.load_der_x509_crl
},
DataType.CSR: {
DataFormat.PEM: x509.load_pem_x509_csr,
DataFormat.DER: x509.load_der_x509_csr
},
})
def collect_default_files(self) -> typing.List[str]:
"""Collect the default files for parsing."""
raise NotImplementedError
def parse_file(self, filename: str) -> typing.List[AuditData]:
"""
Parse a list of AuditData from file.
:param filename: name of the file to parse.
:return list of AuditData parsed from the file.
"""
raise NotImplementedError
def parse_bytes(self, data: bytes):
"""Parse AuditData from bytes."""
for data_type in list(DataType):
try:
result = self.parser[data_type](data)
except ValueError as val_error:
result = None
self.logger.warning(val_error)
if result is not None:
audit_data = AuditData(data_type, result)
return audit_data
return None
def walk_all(self,
results: typing.Dict[str, AuditData],
file_list: typing.Optional[typing.List[str]] = None) \
-> None:
"""
Iterate over all the files in the list and get audit data. The
results will be written to `results` passed to this function.
:param results: The dictionary used to store the parsed
AuditData. The keys of this dictionary should
be the identifier of the AuditData.
"""
if file_list is None:
file_list = self.default_files
for filename in file_list:
data_list = self.parse_file(filename)
for d in data_list:
if d.identifier in results:
results[d.identifier].locations.extend(d.locations)
else:
results[d.identifier] = d
@staticmethod
def find_test_dir():
"""Get the relative path for the Mbed TLS test directory."""
return os.path.relpath(build_tree.guess_mbedtls_root() + '/tests')
class TestDataAuditor(Auditor):
"""Class for auditing files in `framework/data_files/`"""
def collect_default_files(self):
"""Collect all files in `framework/data_files/`"""
test_data_glob = os.path.join(build_tree.guess_mbedtls_root(),
'framework', 'data_files/**')
data_files = [f for f in glob.glob(test_data_glob, recursive=True)
if os.path.isfile(f)]
return data_files
def parse_file(self, filename: str) -> typing.List[AuditData]:
"""
Parse a list of AuditData from data file.
:param filename: name of the file to parse.
:return list of AuditData parsed from the file.
"""
with open(filename, 'rb') as f:
data = f.read()
results = []
# Try to parse all PEM blocks.
is_pem = False
for idx, m in enumerate(re.finditer(X509Parser.PEM_REGEX, data, flags=re.S), 1):
is_pem = True
result = self.parse_bytes(data[m.start():m.end()])
if result is not None:
result.locations.append("{}#{}".format(filename, idx))
results.append(result)
# Might be DER format.
if not is_pem:
result = self.parse_bytes(data)
if result is not None:
result.locations.append("{}".format(filename))
results.append(result)
return results
def parse_suite_data(data_f):
"""
Parses .data file for test arguments that possiblly have a
valid X.509 data. If you need a more precise parser, please
use generate_test_code.parse_test_data instead.
:param data_f: file object of the data file.
:return: Generator that yields test function argument list.
"""
for line in data_f:
line = line.strip()
# Skip comments
if line.startswith('#'):
continue
# Check parameters line
match = re.search(r'\A\w+(.*:)?\"', line)
if match:
# Read test vectors
parts = re.split(r'(?<!\\):', line)
parts = [x for x in parts if x]
args = parts[1:]
yield args
class SuiteDataAuditor(Auditor):
"""Class for auditing files in `tests/suites/*.data`"""
def collect_default_files(self):
"""Collect all files in `tests/suites/*.data`"""
test_dir = self.find_test_dir()
suites_data_folder = os.path.join(test_dir, 'suites')
data_files = glob.glob(os.path.join(suites_data_folder, '*.data'))
return data_files
def parse_file(self, filename: str):
"""
Parse a list of AuditData from test suite data file.
:param filename: name of the file to parse.
:return list of AuditData parsed from the file.
"""
audit_data_list = []
data_f = FileWrapper(filename)
for test_args in parse_suite_data(data_f):
for idx, test_arg in enumerate(test_args):
match = re.match(r'"(?P<data>[0-9a-fA-F]+)"', test_arg)
if not match:
continue
if not X509Parser.check_hex_string(match.group('data')):
continue
audit_data = self.parse_bytes(bytes.fromhex(match.group('data')))
if audit_data is None:
continue
audit_data.locations.append("{}:{}:#{}".format(filename,
data_f.line_no,
idx + 1))
audit_data_list.append(audit_data)
return audit_data_list
def list_all(audit_data: AuditData):
for loc in audit_data.locations:
print("{}\t{:20}\t{:20}\t{:3}\t{}".format(
audit_data.identifier,
audit_data.not_valid_before.isoformat(timespec='seconds'),
audit_data.not_valid_after.isoformat(timespec='seconds'),
audit_data.data_type.name,
loc))
def main():
"""
Perform argument parsing.
"""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('-a', '--all',
action='store_true',
help='list the information of all the files')
parser.add_argument('-v', '--verbose',
action='store_true', dest='verbose',
help='show logs')
parser.add_argument('--from', dest='start_date',
help=('Start of desired validity period (UTC, YYYY-MM-DD). '
'Default: today'),
metavar='DATE')
parser.add_argument('--to', dest='end_date',
help=('End of desired validity period (UTC, YYYY-MM-DD). '
'Default: --from'),
metavar='DATE')
parser.add_argument('--data-files', action='append', nargs='*',
help='data files to audit',
metavar='FILE')
parser.add_argument('--suite-data-files', action='append', nargs='*',
help='suite data files to audit',
metavar='FILE')
args = parser.parse_args()
# start main routine
# setup logger
logger = logging.getLogger()
logging_util.configure_logger(logger)
logger.setLevel(logging.DEBUG if args.verbose else logging.ERROR)
td_auditor = TestDataAuditor(logger)
sd_auditor = SuiteDataAuditor(logger)
data_files = []
suite_data_files = []
if args.data_files is None and args.suite_data_files is None:
data_files = td_auditor.default_files
suite_data_files = sd_auditor.default_files
else:
if args.data_files is not None:
data_files = [x for l in args.data_files for x in l]
if args.suite_data_files is not None:
suite_data_files = [x for l in args.suite_data_files for x in l]
# validity period start date
if args.start_date:
start_date = datetime.datetime.fromisoformat(args.start_date)
else:
start_date = datetime.datetime.today()
# validity period end date
if args.end_date:
end_date = datetime.datetime.fromisoformat(args.end_date)
else:
end_date = start_date
# go through all the files
audit_results = {}
td_auditor.walk_all(audit_results, data_files)
sd_auditor.walk_all(audit_results, suite_data_files)
logger.info("Total: {} objects found!".format(len(audit_results)))
# we filter out the files whose validity duration covers the provided
# duration.
filter_func = lambda d: (start_date < d.not_valid_before) or \
(d.not_valid_after < end_date)
sortby_end = lambda d: d.not_valid_after
if args.all:
filter_func = None
# filter and output the results
for d in sorted(filter(filter_func, audit_results.values()), key=sortby_end):
list_all(d)
logger.debug("Done!")
check_cryptography_version()
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,78 @@
#!/usr/bin/env perl
# Detect comment blocks that are likely meant to be doxygen blocks but aren't.
#
# More precisely, look for normal comment block containing '\'.
# Of course one could use doxygen warnings, eg with:
# sed -e '/EXTRACT/s/YES/NO/' doxygen/mbedtls.doxyfile | doxygen -
# but that would warn about any undocumented item, while our goal is to find
# items that are documented, but not marked as such by mistake.
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use warnings;
use strict;
use File::Basename;
# C/header files in the following directories will be checked
my @mbedtls_directories = qw(include/mbedtls library doxygen/input);
my @tf_psa_crypto_directories = qw(include/psa include/tf-psa-crypto
include/mbedtls
drivers/builtin/include/mbedtls
drivers/builtin/src core dispatch
doxygen/input extras platform utilities);
# very naive pattern to find directives:
# everything with a backslach except '\0' and backslash at EOL
my $doxy_re = qr/\\(?!0|\n)/;
# Return an error code to the environment if a potential error in the
# source code is found.
my $exit_code = 0;
sub check_file {
my ($fname) = @_;
open my $fh, '<', $fname or die "Failed to open '$fname': $!\n";
# first line of the last normal comment block,
# or 0 if not in a normal comment block
my $block_start = 0;
while (my $line = <$fh>) {
$block_start = $. if $line =~ m/\/\*(?![*!])/;
$block_start = 0 if $line =~ m/\*\//;
if ($block_start and $line =~ m/$doxy_re/) {
print "$fname:$block_start: directive on line $.\n";
$block_start = 0; # report only one directive per block
$exit_code = 1;
}
}
close $fh;
}
sub check_dir {
my ($dirname) = @_;
for my $file (<$dirname/*.[ch]>) {
check_file($file);
}
}
open my $project_file, "scripts/project_name.txt" or die "This script must be run from Mbed TLS or TF-PSA-Crypto root directory";
my $project = <$project_file>;
chomp($project);
my @directories;
if ($project eq "TF-PSA-Crypto") {
@directories = @tf_psa_crypto_directories
} elsif ($project eq "Mbed TLS") {
@directories = @mbedtls_directories
}
# Check that the script is being run from the project's root directory.
for my $dir (@directories) {
check_dir($dir)
}
exit $exit_code;
__END__

View File

@@ -0,0 +1,104 @@
#! /usr/bin/env sh
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
# Purpose: check Python files for potential programming errors or maintenance
# hurdles. Run pylint to detect some potential mistakes and enforce PEP8
# coding standards. Run mypy to perform static type checking.
# We'll keep going on errors and report the status at the end.
ret=0
if type python3 >/dev/null 2>/dev/null; then
PYTHON=python3
else
PYTHON=python
fi
check_version () {
$PYTHON - "$2" <<EOF
import packaging.version
import sys
import $1 as package
actual = package.__version__
wanted = sys.argv[1]
if packaging.version.parse(actual) < packaging.version.parse(wanted):
sys.stderr.write("$1: version %s is too old (want %s)\n" % (actual, wanted))
exit(1)
EOF
}
can_pylint () {
# Pylint 1.5.2 from Ubuntu 16.04 is too old:
# E: 34, 0: Unable to import 'mbedtls_framework' (import-error)
# Pylint 1.8.3 from Ubuntu 18.04 passed on the first commit containing this line.
check_version pylint 1.8.3
}
can_mypy () {
# mypy 0.770 is too old:
# framework/scripts/test_psa_constant_names.py:34: error: Cannot find implementation or library stub for module named 'mbedtls_framework'
# mypy 0.780 from pip passed on the first commit containing this line.
check_version mypy.version 0.780
}
# With just a --can-xxx option, check whether the tool for xxx is available
# with an acceptable version, and exit without running any checks. The exit
# status is true if the tool is available and acceptable and false otherwise.
if [ "$1" = "--can-pylint" ]; then
can_pylint
exit
elif [ "$1" = "--can-mypy" ]; then
can_mypy
exit
fi
echo 'Running pylint ...'
# Exclude `maintainer` subdirectories, because they can contain code
# that does not work with the versions of pylint and mypy we use on the CI.
# https://github.com/Mbed-TLS/mbedtls-framework/issues/293
#
# When we move Python code between repositories, there is a transition
# period during which code is duplicated between the old repository and
# the new repository.
# Pylint looks for duplicate code inside files that are mentioned in the
# same invocation. When we move some code from A to B, we want to skip
# duplicate-code checks between A and B. So we arrange for two separate
# runs of pylint: one for the A files, and one for the others.
# Remove exceptions below once the A file (or the moved code in the A file)
# has been removed from all consuming branches.
find framework/scripts scripts tests/scripts \
-name maintainer -prune -o \
-name '*.py' \( \
! -path scripts/abi_check.py \
! -path scripts/code_size_compare.py \
! -path scripts/ecp_comb_table.py \
! -path tests/scripts/audit-validity-dates.py \
! -path tests/scripts/generate_server9_bad_saltlen.py \
! -path tests/scripts/psa_collect_statuses.py \
! -path tests/scripts/run_demos.py \
! -path tests/scripts/test_config_script.py \
! -path framework/scripts/make_generated_files.py \
-exec $PYTHON -m pylint {} + \
-o -exec $PYTHON -m pylint {} + \) || {
echo >&2 "pylint reported errors"
ret=1
}
echo
echo 'Running mypy ...'
$PYTHON -m mypy framework/scripts || {
echo >&2 "mypy reported errors in the framework"
ret=1
}
# Exclude `maintainer` subdirectories, because they can contain code
# that does not work with the versions of pylint and mypy we use on the CI.
# https://github.com/Mbed-TLS/mbedtls-framework/issues/293
$PYTHON -m mypy --exclude maintainer scripts tests/scripts || {
echo >&2 "mypy reported errors in the parent repository"
ret=1
}
exit $ret

View File

@@ -0,0 +1,575 @@
#!/usr/bin/env python3
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
"""
This script checks the current state of the source code for minor issues,
including incorrect file permissions, presence of tabs, non-Unix line endings,
trailing whitespace, and presence of UTF-8 BOM.
Note: requires python 3, must be run from Mbed TLS root.
"""
import argparse
import codecs
import inspect
import logging
import os
import re
import subprocess
import sys
try:
from typing import FrozenSet, Optional, Pattern # pylint: disable=unused-import
except ImportError:
pass
from mbedtls_framework import build_tree
class FileIssueTracker:
"""Base class for file-wide issue tracking.
To implement a checker that processes a file as a whole, inherit from
this class and implement `check_file_for_issue` and define ``heading``.
``suffix_exemptions``: files whose name ends with a string in this set
will not be checked.
``path_exemptions``: files whose path (relative to the root of the source
tree) matches this regular expression will not be checked. This can be
``None`` to match no path. Paths are normalized and converted to ``/``
separators before matching.
``heading``: human-readable description of the issue
"""
suffix_exemptions = frozenset() #type: FrozenSet[str]
path_exemptions = None #type: Optional[Pattern[str]]
# heading must be defined in derived classes.
# pylint: disable=no-member
def __init__(self):
self.files_with_issues = {}
@staticmethod
def normalize_path(filepath):
"""Normalize ``filepath`` with / as the directory separator."""
filepath = os.path.normpath(filepath)
# On Windows, we may have backslashes to separate directories.
# We need slashes to match exemption lists.
seps = os.path.sep
if os.path.altsep is not None:
seps += os.path.altsep
return '/'.join(filepath.split(seps))
def should_check_file(self, filepath):
"""Whether the given file name should be checked.
Files whose name ends with a string listed in ``self.suffix_exemptions``
or whose path matches ``self.path_exemptions`` will not be checked.
"""
for files_exemption in self.suffix_exemptions:
if filepath.endswith(files_exemption):
return False
if self.path_exemptions and \
re.match(self.path_exemptions, self.normalize_path(filepath)):
return False
return True
def check_file_for_issue(self, filepath):
"""Check the specified file for the issue that this class is for.
Subclasses must implement this method.
"""
raise NotImplementedError
def record_issue(self, filepath, line_number):
"""Record that an issue was found at the specified location."""
if filepath not in self.files_with_issues.keys():
self.files_with_issues[filepath] = []
self.files_with_issues[filepath].append(line_number)
def output_file_issues(self, logger):
"""Log all the locations where the issue was found."""
if self.files_with_issues.values():
logger.info(self.heading)
for filename, lines in sorted(self.files_with_issues.items()):
if lines:
logger.info("{}: {}".format(
filename, ", ".join(str(x) for x in lines)
))
else:
logger.info(filename)
logger.info("")
BINARY_FILE_PATH_RE_LIST = [
r'docs/.*\.pdf\Z',
r'docs/.*\.png\Z',
r'tf-psa-crypto/docs/.*\.pdf\Z',
r'tf-psa-crypto/docs/.*\.png\Z',
r'programs/fuzz/corpuses/[^.]+\Z',
r'framework/data_files/[^.]+\Z',
r'framework/data_files/.*\.(crt|csr|db|der|key|pubkey)\Z',
r'framework/data_files/.*\.req\.[^/]+\Z',
r'framework/data_files/.*malformed[^/]+\Z',
r'framework/data_files/format_pkcs12\.fmt\Z',
r'framework/data_files/.*\.bin\Z',
]
BINARY_FILE_PATH_RE = re.compile('|'.join(BINARY_FILE_PATH_RE_LIST))
class LineIssueTracker(FileIssueTracker):
"""Base class for line-by-line issue tracking.
To implement a checker that processes files line by line, inherit from
this class and implement `line_with_issue`.
"""
# Exclude binary files.
path_exemptions = BINARY_FILE_PATH_RE
def issue_with_line(self, line, filepath, line_number):
"""Check the specified line for the issue that this class is for.
Subclasses must implement this method.
"""
raise NotImplementedError
def check_file_line(self, filepath, line, line_number):
if self.issue_with_line(line, filepath, line_number):
self.record_issue(filepath, line_number)
def check_file_for_issue(self, filepath):
"""Check the lines of the specified file.
Subclasses must implement the ``issue_with_line`` method.
"""
with open(filepath, "rb") as f:
for i, line in enumerate(iter(f.readline, b"")):
self.check_file_line(filepath, line, i + 1)
def is_windows_file(filepath):
_root, ext = os.path.splitext(filepath)
return ext in ('.bat', '.dsp', '.dsw', '.sln', '.vcxproj')
class ShebangIssueTracker(FileIssueTracker):
"""Track files with a bad, missing or extraneous shebang line.
Executable scripts must start with a valid shebang (#!) line.
"""
heading = "Invalid shebang line:"
# Allow either /bin/sh, /bin/bash, or /usr/bin/env.
# Allow at most one argument (this is a Linux limitation).
# For sh and bash, the argument if present must be options.
# For env, the argument must be the base name of the interpreter.
_shebang_re = re.compile(rb'^#! ?(?:/bin/(bash|sh)(?: -[^\n ]*)?'
rb'|/usr/bin/env ([^\n /]+))$')
_extensions = {
b'bash': 'sh',
b'perl': 'pl',
b'python3': 'py',
b'sh': 'sh',
}
path_exemptions = re.compile(r'framework/scripts/quiet/.*')
def is_valid_shebang(self, first_line, filepath):
m = re.match(self._shebang_re, first_line)
if not m:
return False
interpreter = m.group(1) or m.group(2)
if interpreter not in self._extensions:
return False
if not filepath.endswith('.' + self._extensions[interpreter]):
return False
return True
def check_file_for_issue(self, filepath):
is_executable = os.access(filepath, os.X_OK)
with open(filepath, "rb") as f:
first_line = f.readline()
if first_line.startswith(b'#!'):
if not is_executable:
# Shebang on a non-executable file
self.files_with_issues[filepath] = None
elif not self.is_valid_shebang(first_line, filepath):
self.files_with_issues[filepath] = [1]
elif is_executable:
# Executable without a shebang
self.files_with_issues[filepath] = None
class EndOfFileNewlineIssueTracker(FileIssueTracker):
"""Track files that end with an incomplete line
(no newline character at the end of the last line)."""
heading = "Missing newline at end of file:"
path_exemptions = BINARY_FILE_PATH_RE
def check_file_for_issue(self, filepath):
with open(filepath, "rb") as f:
try:
f.seek(-1, 2)
except OSError:
# This script only works on regular files. If we can't seek
# 1 before the end, it means that this position is before
# the beginning of the file, i.e. that the file is empty.
return
if f.read(1) != b"\n":
self.files_with_issues[filepath] = None
class Utf8BomIssueTracker(FileIssueTracker):
"""Track files that start with a UTF-8 BOM.
Files should be ASCII or UTF-8. Valid UTF-8 does not start with a BOM."""
heading = "UTF-8 BOM present:"
suffix_exemptions = frozenset([".vcxproj", ".sln"])
path_exemptions = BINARY_FILE_PATH_RE
def check_file_for_issue(self, filepath):
with open(filepath, "rb") as f:
if f.read().startswith(codecs.BOM_UTF8):
self.files_with_issues[filepath] = None
class UnicodeIssueTracker(LineIssueTracker):
"""Track lines with invalid characters or invalid text encoding."""
heading = "Invalid UTF-8 or forbidden character:"
# Only allow valid UTF-8, and only other explicitly allowed characters.
# We deliberately exclude all characters that aren't a simple non-blank,
# non-zero-width glyph, apart from a very small set (tab, ordinary space,
# line breaks, "basic" no-break space and soft hyphen). In particular,
# non-ASCII control characters, combinig characters, and Unicode state
# changes (e.g. right-to-left text) are forbidden.
# Note that we do allow some characters with a risk of visual confusion,
# for example '-' (U+002D HYPHEN-MINUS) vs '­' (U+00AD SOFT HYPHEN) vs
# '' (U+2010 HYPHEN), or 'A' (U+0041 LATIN CAPITAL LETTER A) vs
# 'Α' (U+0391 GREEK CAPITAL LETTER ALPHA).
GOOD_CHARACTERS = ''.join([
'\t\n\r -~', # ASCII (tabs and line endings are checked separately)
'\u00A0-\u00FF', # Latin-1 Supplement (for NO-BREAK SPACE and punctuation)
'\u2010-\u2027\u2030-\u205E', # General Punctuation (printable)
'\u2070\u2071\u2074-\u208E\u2090-\u209C', # Superscripts and Subscripts
'\u2190-\u21FF', # Arrows
'\u2200-\u22FF', # Mathematical Symbols
'\u2500-\u257F' # Box Drawings characters used in markdown trees
])
# Allow any of the characters and ranges above, and anything classified
# as a word constituent.
GOOD_CHARACTERS_RE = re.compile(r'[\w{}]+\Z'.format(GOOD_CHARACTERS))
def issue_with_line(self, line, _filepath, line_number):
try:
text = line.decode('utf-8')
except UnicodeDecodeError:
return True
if line_number == 1 and text.startswith('\uFEFF'):
# Strip BOM (U+FEFF ZERO WIDTH NO-BREAK SPACE) at the beginning.
# Which files are allowed to have a BOM is handled in
# Utf8BomIssueTracker.
text = text[1:]
return not self.GOOD_CHARACTERS_RE.match(text)
class UnixLineEndingIssueTracker(LineIssueTracker):
"""Track files with non-Unix line endings (i.e. files with CR)."""
heading = "Non-Unix line endings:"
def should_check_file(self, filepath):
if not super().should_check_file(filepath):
return False
return not is_windows_file(filepath)
def issue_with_line(self, line, _filepath, _line_number):
return b"\r" in line
class WindowsLineEndingIssueTracker(LineIssueTracker):
"""Track files with non-Windows line endings (i.e. CR or LF not in CRLF)."""
heading = "Non-Windows line endings:"
def should_check_file(self, filepath):
if not super().should_check_file(filepath):
return False
return is_windows_file(filepath)
def issue_with_line(self, line, _filepath, _line_number):
return not line.endswith(b"\r\n") or b"\r" in line[:-2]
class TrailingWhitespaceIssueTracker(LineIssueTracker):
"""Track lines with trailing whitespace."""
heading = "Trailing whitespace:"
suffix_exemptions = frozenset([".diff", ".dsp", ".md", ".patch"])
def issue_with_line(self, line, _filepath, _line_number):
return line.rstrip(b"\r\n") != line.rstrip()
class TabIssueTracker(LineIssueTracker):
"""Track lines with tabs."""
heading = "Tabs present:"
suffix_exemptions = frozenset([
".make",
".pem", # some openssl dumps have tabs
".sln",
"/.gitmodules",
"/Makefile",
"/Makefile.inc",
"/generate_visualc_files.pl",
])
def issue_with_line(self, line, _filepath, _line_number):
return b"\t" in line
class MergeArtifactIssueTracker(LineIssueTracker):
"""Track lines with merge artifacts.
These are leftovers from a ``git merge`` that wasn't fully edited."""
heading = "Merge artifact:"
def issue_with_line(self, line, _filepath, _line_number):
# Detect leftover git conflict markers.
if line.startswith(b'<<<<<<< ') or line.startswith(b'>>>>>>> '):
return True
if line.startswith(b'||||||| '): # from merge.conflictStyle=diff3
return True
if line.rstrip(b'\r\n') == b'=======' and \
not _filepath.endswith('.md'):
return True
return False
def this_location():
frame = inspect.currentframe()
assert frame is not None
info = inspect.getframeinfo(frame)
return os.path.basename(info.filename), info.lineno
THIS_FILE_BASE_NAME, LINE_NUMBER_BEFORE_LICENSE_ISSUE_TRACKER = this_location()
class LicenseIssueTracker(LineIssueTracker):
"""Check copyright statements and license indications.
This class only checks that statements are correct if present. It does
not enforce the presence of statements in each file.
"""
heading = "License issue:"
LICENSE_EXEMPTION_RE_LIST = []
# Exempt third-party drivers which may be under a different license
if build_tree.looks_like_tf_psa_crypto_root(os.getcwd()):
LICENSE_EXEMPTION_RE_LIST.append(r'drivers/(?=(everest)/.*)')
elif build_tree.is_mbedtls_3_6():
LICENSE_EXEMPTION_RE_LIST.append(r'3rdparty/(?!(p256-m)/.*)')
LICENSE_EXEMPTION_RE_LIST += [
# Documentation explaining the license may have accidental
# false positives.
r'(ChangeLog|LICENSE|framework\/LICENSE|[-0-9A-Z_a-z]+\.md)\Z',
# Files imported from TF-M, and not used except in test builds,
# may be under a different license.
r'configs/ext/crypto_config_profile_medium\.h\Z',
r'configs/ext/tfm_mbedcrypto_config_profile_medium\.h\Z',
r'configs/ext/README\.md\Z',
# Third-party file.
r'dco\.txt\Z',
r'framework\/dco\.txt\Z',
]
path_exemptions = re.compile('|'.join(BINARY_FILE_PATH_RE_LIST +
LICENSE_EXEMPTION_RE_LIST))
COPYRIGHT_HOLDER = rb'The Mbed TLS Contributors'
# Catch "Copyright foo", "Copyright (C) foo", "Copyright © foo", etc.
COPYRIGHT_RE = re.compile(rb'.*\bcopyright\s+((?:\w|\s|[()]|[^ -~])*\w)', re.I)
SPDX_HEADER_KEY = b'SPDX-License-Identifier'
LICENSE_IDENTIFIER = b'Apache-2.0 OR GPL-2.0-or-later'
SPDX_RE = re.compile(br'.*?(' +
re.escape(SPDX_HEADER_KEY) +
br')(:\s*(.*?)\W*\Z|.*)', re.I)
LICENSE_MENTION_RE = re.compile(rb'.*(?:' + rb'|'.join([
rb'Apache License',
rb'General Public License',
]) + rb')', re.I)
def __init__(self):
super().__init__()
# Record what problem was caused. We can't easily report it due to
# the structure of the script. To be fixed after
# https://github.com/Mbed-TLS/mbedtls/pull/2506
self.problem = None
def issue_with_line(self, line, filepath, line_number):
#pylint: disable=too-many-return-statements
# Use endswith() rather than the more correct os.path.basename()
# because experimentally, it makes a significant difference to
# the running time.
if filepath.endswith(THIS_FILE_BASE_NAME) and \
line_number > LINE_NUMBER_BEFORE_LICENSE_ISSUE_TRACKER:
# Avoid false positives from the code in this class.
# Also skip the rest of this file, which is highly unlikely to
# contain any problematic statements since we put those near the
# top of files.
return False
m = self.COPYRIGHT_RE.match(line)
if m and m.group(1) != self.COPYRIGHT_HOLDER:
self.problem = 'Invalid copyright line'
return True
m = self.SPDX_RE.match(line)
if m:
if m.group(1) != self.SPDX_HEADER_KEY:
self.problem = 'Misspelled ' + self.SPDX_HEADER_KEY.decode()
return True
if not m.group(3):
self.problem = 'Improperly formatted SPDX license identifier'
return True
if m.group(3) != self.LICENSE_IDENTIFIER:
self.problem = 'Wrong SPDX license identifier'
return True
m = self.LICENSE_MENTION_RE.match(line)
if m:
self.problem = 'Suspicious license mention'
return True
return False
class ErrorAddIssueTracker(LineIssueTracker):
"""Signal direct additions of error codes.
Adding a low-level error code with a high-level error code is deprecated
and should use MBEDTLS_ERROR_ADD.
"""
heading = "Direct addition of error codes"
_ERR_PLUS_RE = re.compile(br'MBEDTLS_ERR_\w+ *\+|'
br'\+ *MBEDTLS_ERR_')
_EXCLUDE_RE = re.compile(br' *case ')
def issue_with_line(self, line, filepath, line_number):
if self._ERR_PLUS_RE.search(line) and not self._EXCLUDE_RE.match(line):
return True
return False
class IntegrityChecker:
"""Sanity-check files under the current directory."""
def __init__(self, log_file):
"""Instantiate the sanity checker.
Check files under the current directory.
Write a report of issues to log_file."""
if not build_tree.looks_like_root(os.getcwd()):
raise Exception("This script must be run from Mbed TLS or TF-PSA-Crypto root")
self.logger = None
self.setup_logger(log_file)
self.issues_to_check = [
ShebangIssueTracker(),
EndOfFileNewlineIssueTracker(),
Utf8BomIssueTracker(),
UnicodeIssueTracker(),
UnixLineEndingIssueTracker(),
WindowsLineEndingIssueTracker(),
TrailingWhitespaceIssueTracker(),
TabIssueTracker(),
MergeArtifactIssueTracker(),
LicenseIssueTracker(),
]
if not build_tree.is_mbedtls_3_6():
self.issues_to_check.append(ErrorAddIssueTracker())
def setup_logger(self, log_file, level=logging.INFO):
"""Log to log_file if provided, or to stderr if None."""
self.logger = logging.getLogger()
self.logger.setLevel(level)
if log_file:
handler = logging.FileHandler(log_file)
self.logger.addHandler(handler)
else:
console = logging.StreamHandler()
self.logger.addHandler(console)
@staticmethod
def collect_files():
"""Return the list of files to check.
These are the regular files commited into Git.
"""
bytes_output = subprocess.check_output(['git', '-C', 'framework',
'ls-files', '-z'])
bytes_framework_filepaths = bytes_output.split(b'\0')[:-1]
bytes_framework_filepaths = ["framework/".encode() + filepath
for filepath in bytes_framework_filepaths]
bytes_output = subprocess.check_output(['git', 'ls-files', '-z'])
bytes_filepaths = bytes_output.split(b'\0')[:-1] + \
bytes_framework_filepaths
ascii_filepaths = map(lambda fp: fp.decode('ascii'), bytes_filepaths)
# Filter out directories. Normally Git doesn't list directories
# (it only knows about the files inside them), but there is
# at least one case where 'git ls-files' includes a directory:
# submodules. Just skip submodules (and any other directories).
ascii_filepaths = [fp for fp in ascii_filepaths
if os.path.isfile(fp)]
# Prepend './' to files in the top-level directory so that
# something like `'/Makefile' in fp` matches in the top-level
# directory as well as in subdirectories.
return [fp if os.path.dirname(fp) else os.path.join(os.curdir, fp)
for fp in ascii_filepaths]
def check_files(self):
"""Check all files for all issues."""
for issue_to_check in self.issues_to_check:
for filepath in self.collect_files():
if issue_to_check.should_check_file(filepath):
issue_to_check.check_file_for_issue(filepath)
def output_issues(self):
"""Log the issues found and their locations.
Return 1 if there were issues, 0 otherwise.
"""
integrity_return_code = 0
for issue_to_check in self.issues_to_check:
if issue_to_check.files_with_issues:
integrity_return_code = 1
issue_to_check.output_file_issues(self.logger)
return integrity_return_code
def run_main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument(
"-l", "--log_file", type=str, help="path to optional output log",
)
check_args = parser.parse_args()
integrity_check = IntegrityChecker(check_args.log_file)
integrity_check.check_files()
return_code = integrity_check.output_issues()
sys.exit(return_code)
if __name__ == "__main__":
run_main()

1328
vendor/mbedtls/framework/scripts/check_names.py vendored Executable file

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,82 @@
#!/usr/bin/env python3
"""Sanity checks for test data."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import re
import sys
from mbedtls_framework import collect_test_cases
class DescriptionChecker(collect_test_cases.TestDescriptionExplorer):
"""Check all test case descriptions.
* Check that each description is valid (length, allowed character set, etc.).
* Check that there is no duplicated description inside of one test suite.
"""
def __init__(self, results):
self.results = results
def new_per_file_state(self):
"""Dictionary mapping descriptions to their line number."""
return {}
def process_test_case(self, per_file_state,
file_name, line_number, description):
"""Check test case descriptions for errors."""
results = self.results
seen = per_file_state
if description in seen:
results.error(file_name, line_number,
'Duplicate description (also line {})',
seen[description])
return
if re.search(br'[\t;]', description):
results.error(file_name, line_number,
'Forbidden character \'{}\' in description',
re.search(br'[\t;]', description).group(0).decode('ascii'))
if re.search(br'[^ -~]', description):
results.error(file_name, line_number,
'Non-ASCII character in description')
if len(description) > 66:
results.warning(file_name, line_number,
'Test description too long ({} > 66)',
len(description))
seen[description] = line_number
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--list-all',
action='store_true',
help='List all test cases, without doing checks')
parser.add_argument('--quiet', '-q',
action='store_true',
help='Hide warnings')
parser.add_argument('--verbose', '-v',
action='store_false', dest='quiet',
help='Show warnings (default: on; undoes --quiet)')
options = parser.parse_args()
if options.list_all:
descriptions = collect_test_cases.collect_available_test_cases()
sys.stdout.write('\n'.join(descriptions + ['']))
return
results = collect_test_cases.Results(options)
checker = DescriptionChecker(results)
try:
checker.walk_all()
except collect_test_cases.ScriptOutputError as e:
results.error(e.script_name, e.idx,
'"{}" should be listed as "<suite_name>;<description>"',
e.line)
if (results.warnings or results.errors) and not options.quiet:
sys.stderr.write('{}: {} errors, {} warnings\n'
.format(sys.argv[0], results.errors, results.warnings))
sys.exit(1 if results.errors else 0)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,11 @@
# Python package requirements for Mbed TLS testing.
# At the time of writing, only needed for scripts/audit-validity-dates.py.
# It needs >=35.0.0 for correct operation, and that requires Python >=3.6.
# >=35.0.0 also requires Rust to build from source, which we are forced to do on
# FreeBSD, since PyPI doesn't carry binary wheels for the BSDs.
cryptography >= 35.0.0; platform_system == 'Linux'
# At the time of writing, only needed for
# scripts/generate_server9_bad_saltlen.py.
asn1crypto; platform_system == 'Linux'

306
vendor/mbedtls/framework/scripts/code_style.py vendored Executable file
View File

@@ -0,0 +1,306 @@
#!/usr/bin/env python3
"""Check or fix the code style by running Uncrustify.
This script must be run from the root of a Git work tree containing Mbed TLS.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import os
import re
import subprocess
import sys
from typing import FrozenSet, List, Optional
from mbedtls_framework import build_tree
UNCRUSTIFY_SUPPORTED_VERSION = "0.75.1"
CONFIG_FILE = ".uncrustify.cfg"
UNCRUSTIFY_ARGS = ["-c", CONFIG_FILE]
CHECK_GENERATED_FILES = "tests/scripts/check-generated-files.sh"
def print_err(*args):
print("Error: ", *args, file=sys.stderr)
# Print the file names that will be skipped and the help message
def print_skip(files_to_skip):
print()
print(*files_to_skip, sep=", SKIP\n", end=", SKIP\n")
print("Warning: The listed files will be skipped because\n"
"they are not known to git.")
print()
# Match FILENAME(s) in "check SCRIPT (FILENAME...)"
CHECK_CALL_RE = re.compile(r"\n\s*check\s+[^\s#$&*?;|]+([^\n#$&*?;|]+)",
re.ASCII)
def list_generated_files() -> FrozenSet[str]:
"""Return the names of generated files.
We don't reformat generated files, since the result might be different
from the output of the generator. Ideally the result of the generator
would conform to the code style, but this would be difficult, especially
with respect to the placement of line breaks in long logical lines.
"""
if build_tree.is_mbedtls_3_6():
# Parse check-generated-files.sh to get an up-to-date list of
# generated files. Read the file rather than calling it so that
# this script only depends on Git, Python and uncrustify, and not other
# tools such as sh or grep which might not be available on Windows.
# This introduces a limitation: check-generated-files.sh must have
# the expected format and must list the files explicitly, not through
# wildcards or command substitution.
content = open(CHECK_GENERATED_FILES, encoding="utf-8").read()
checks = re.findall(CHECK_CALL_RE, content)
return frozenset(word for s in checks for word in s.split())
else:
output = subprocess.check_output(["framework/scripts/make_generated_files.py",
"--list"], universal_newlines=True)
# psa_test_wrappers.[hc], generated by generate_psa_wrappers.py, are
# currently committed and unknown to make_generated_files.py. Add them
# here to the list of generated file as we do not want to check their
# coding style.
if build_tree.looks_like_tf_psa_crypto_root("."):
output += "tests/include/test/psa_test_wrappers.h\n"
output += "tests/src/psa_test_wrappers.c"
return frozenset(line for line in output.splitlines())
# Check for comment string indicating an auto-generated file
AUTOGEN_RE = re.compile(r"Warning[ :-]+This file is (now )?auto[ -]?generated",
re.ASCII | re.IGNORECASE)
def is_file_autogenerated(filename):
content = open(filename, encoding="utf-8").read()
return AUTOGEN_RE.search(content) is not None
def get_src_files(since: Optional[str]) -> List[str]:
"""
Use git to get a list of the source files.
The optional argument since is a commit, indicating to only list files
that have changed since that commit. Without this argument, list all
files known to git.
Only C files are included, and certain files (generated, or third party)
are excluded.
"""
file_patterns = ["*.[hc]",
"tests/suites/*.function",
"scripts/data_files/*.fmt"]
output = subprocess.check_output(["git", "ls-files"] + file_patterns,
universal_newlines=True)
src_files = output.split()
# When this script is called from a git hook, some environment variables
# are set by default which force all git commands to use the main repository
# (i.e. prevent us from performing commands on the framework repo).
# Create an environment without these variables for running commands on the
# framework repo.
framework_env = os.environ.copy()
# Get a list of environment vars that git sets
git_env_vars = subprocess.check_output(["git", "rev-parse", "--local-env-vars"],
universal_newlines=True)
# Remove the vars from the environment
for var in git_env_vars.split():
framework_env.pop(var, None)
output = subprocess.check_output(["git", "-C", "framework", "ls-files"]
+ file_patterns,
universal_newlines=True,
env=framework_env)
framework_src_files = output.split()
if since:
# get all files changed in commits since the starting point in ...
# ... the main repository
cmd = ["git", "log", since + "..HEAD", "--ignore-submodules",
"--name-only", "--pretty=", "--"] + src_files
output = subprocess.check_output(cmd, universal_newlines=True)
committed_changed_files = output.split()
# ... the framework submodule
framework_since = get_submodule_hash(since, "framework")
cmd = ["git", "-C", "framework", "log", framework_since + "..HEAD",
"--name-only", "--pretty=", "--"] + framework_src_files
output = subprocess.check_output(cmd, universal_newlines=True,
env=framework_env)
committed_changed_files += ["framework/" + s for s in output.split()]
# and also get all files with uncommitted changes in ...
# ... the main repository
cmd = ["git", "diff", "--name-only", "--"] + src_files
output = subprocess.check_output(cmd, universal_newlines=True)
uncommitted_changed_files = output.split()
# ... the framework submodule
cmd = ["git", "-C", "framework", "diff", "--name-only", "--"] + \
framework_src_files
output = subprocess.check_output(cmd, universal_newlines=True,
env=framework_env)
uncommitted_changed_files += ["framework/" + s for s in output.split()]
src_files = committed_changed_files + uncommitted_changed_files
else:
src_files += ["framework/" + s for s in framework_src_files]
generated_files = list_generated_files()
# Don't correct style for third-party files (and, for simplicity,
# companion files in the same subtree), or for automatically
# generated files (we're correcting the templates instead).
if build_tree.is_mbedtls_3_6():
src_files = [filename for filename in src_files
if not (filename.startswith("3rdparty/") or
filename in generated_files or
is_file_autogenerated(filename))]
else:
src_files = [filename for filename in src_files
if not (filename.startswith("drivers/everest/") or
filename.startswith("drivers/p256-m/") or
filename in generated_files or
is_file_autogenerated(filename))]
return src_files
def get_submodule_hash(commit: str, submodule: str) -> str:
"""Get the commit hash of a submodule at a given commit in the Git repository."""
cmd = ["git", "ls-tree", commit, submodule]
output = subprocess.check_output(cmd, universal_newlines=True)
return output.split()[2]
def get_uncrustify_version(uncrustify_exe: str) -> str:
"""
Get the version string from Uncrustify.
Return an empty string if Uncrustify is not found.
"""
try:
output = subprocess.check_output([uncrustify_exe, "--version"],
stderr=subprocess.PIPE)
return str(output, "utf-8").strip()
except FileNotFoundError:
sys.stderr.write('Fatal: command {} not found in PATH.\n'
.format(uncrustify_exe))
return ''
def check_style_is_correct(uncrustify_exe: str,
src_file_list: List[str]) -> bool:
"""
Check the code style and output a diff for each file whose style is
incorrect.
"""
style_correct = True
for src_file in src_file_list:
uncrustify_cmd = [uncrustify_exe] + UNCRUSTIFY_ARGS + [src_file]
result = subprocess.run(uncrustify_cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE, check=False)
if result.returncode != 0:
print_err("Uncrustify returned " + str(result.returncode) +
" correcting file " + src_file)
return False
# Uncrustify makes changes to the code and places the result in a new
# file with the extension ".uncrustify". To get the changes (if any)
# simply diff the 2 files.
diff_cmd = ["diff", "-u", src_file, src_file + ".uncrustify"]
cp = subprocess.run(diff_cmd, check=False)
if cp.returncode == 1:
print(src_file + " changed - code style is incorrect.")
style_correct = False
elif cp.returncode != 0:
raise subprocess.CalledProcessError(cp.returncode, cp.args,
cp.stdout, cp.stderr)
# Tidy up artifact
os.remove(src_file + ".uncrustify")
return style_correct
def fix_style_single_pass(uncrustify_exe: str,
src_file_list: List[str]) -> bool:
"""
Run Uncrustify once over the source files.
"""
code_change_args = UNCRUSTIFY_ARGS + ["--no-backup"]
for src_file in src_file_list:
uncrustify_cmd = [uncrustify_exe] + code_change_args + [src_file]
result = subprocess.run(uncrustify_cmd, check=False)
if result.returncode != 0:
print_err("Uncrustify with file returned: " +
str(result.returncode) + " correcting file " +
src_file)
return False
return True
def fix_style(uncrustify_exe: str, src_file_list: List[str]) -> int:
"""
Fix the code style. This takes 2 passes of Uncrustify.
"""
if not fix_style_single_pass(uncrustify_exe, src_file_list):
return 1
if not fix_style_single_pass(uncrustify_exe, src_file_list):
return 1
# Guard against future changes that cause the codebase to require
# more passes.
if not check_style_is_correct(uncrustify_exe, src_file_list):
print_err("Code style still incorrect after second run of Uncrustify.")
return 1
else:
return 0
def main() -> int:
"""
Main with command line arguments.
"""
parser = argparse.ArgumentParser()
parser.add_argument('-f', '--fix', action='store_true',
help=('modify source files to fix the code style '
'(default: print diff, do not modify files)'))
parser.add_argument('-s', '--since', metavar='COMMIT', const='development', nargs='?',
help=('only check files modified since the specified commit'
' (e.g. --since=HEAD~3 or --since=development). If no'
' commit is specified, default to development.'))
# --subset is almost useless: it only matters if there are no files
# ('code_style.py' without arguments checks all files known to Git,
# 'code_style.py --subset' does nothing). In particular,
# 'code_style.py --fix --subset ...' is intended as a stable ("porcelain")
# way to restyle a possibly empty set of files.
parser.add_argument('--subset', action='store_true',
help='only check the specified files (default with non-option arguments)')
parser.add_argument('--uncrustify',
default='uncrustify',
help='uncrustify command to run (default: uncrustify)')
parser.add_argument('operands', nargs='*', metavar='FILE',
help='files to check (files MUST be known to git, if none: check all)')
args = parser.parse_args()
uncrustify_version = get_uncrustify_version(args.uncrustify)
if UNCRUSTIFY_SUPPORTED_VERSION not in uncrustify_version:
if uncrustify_version != '':
sys.stderr.write('Fatal: wrong uncrustify version ({}).\n'
.format(uncrustify_version))
sys.stderr.write('You need uncrustify {} for correct results.\n'
.format(UNCRUSTIFY_SUPPORTED_VERSION))
return 2
covered = frozenset(get_src_files(args.since))
# We only check files that are known to git
if args.subset or args.operands:
src_files = [f for f in args.operands if f in covered]
skip_src_files = [f for f in args.operands if f not in covered]
if skip_src_files:
print_skip(skip_src_files)
else:
src_files = list(covered)
if args.fix:
# Fix mode
return fix_style(args.uncrustify, src_files)
else:
# Check mode
if check_style_is_correct(args.uncrustify, src_files):
print("Checked {} files, style ok.".format(len(src_files)))
return 0
else:
return 1
if __name__ == '__main__':
sys.exit(main())

View File

@@ -0,0 +1,181 @@
## Common shell functions used by demo scripts programs/*/*.sh.
## How to write a demo script
## ==========================
##
## Include this file near the top of each demo script:
## . "${0%/*}/demo_common.sh"
##
## Start with a "msg" call that explains the purpose of the script.
## Then call the "depends_on" function to ensure that all config
## dependencies are met.
##
## As the last thing in the script, call the cleanup function.
##
## You can use the functions and variables described below.
set -e -u
# Check if the provided path ($1) can be a valid root for Mbed TLS or TF-PSA-Crypto.
# This is based on the fact that "scripts/project_name.txt" exists.
is_valid_root () {
if ! [ -f "$1/scripts/project_name.txt" ]; then
return 1;
fi
return 0;
}
## At the end of the while loop below $root_dir will point to the root directory
## of the Mbed TLS or TF-PSA-Crypto source tree.
root_dir="${0%/*}"
## Find a nice path to the root directory, avoiding unnecessary "../".
##
## The code supports demo scripts nested up to 4 levels deep.
##
## The code works no matter where the demo script is relative to the current
## directory, even if it is called with a relative path.
n=4
while true; do
# If we went up too many folders, then give up and return a failure.
if [ $n -eq 0 ]; then
echo >&2 "This doesn't seem to be an Mbed TLS source tree."
exit 125
fi
if is_valid_root "$root_dir"; then
break;
fi
n=$((n - 1))
case $root_dir in
.) root_dir="..";;
..|?*/..) root_dir="$root_dir/..";;
?*/*) root_dir="${root_dir%/*}";;
/*) root_dir="/";;
*) root_dir=".";;
esac
done
# Now that we have a root path we can source the "project_detection.sh" script.
. "$root_dir/framework/scripts/project_detection.sh"
## msg LINE...
## msg <TEXT_ORIGIN
## Display an informational message.
msg () {
if [ $# -eq 0 ]; then
sed 's/^/# /'
else
for x in "$@"; do
echo "# $x"
done
fi
}
## run "Message" COMMAND ARGUMENT...
## Display the message, then run COMMAND with the specified arguments.
run () {
echo
echo "# $1"
shift
echo "+ $*"
"$@"
}
## Like '!', but stop on failure with 'set -e'
not () {
if "$@"; then false; fi
}
## run_bad "Message" COMMAND ARGUMENT...
## Like run, but the command is expected to fail.
run_bad () {
echo
echo "$1 This must fail."
shift
echo "+ ! $*"
not "$@"
}
## This check is temporary and it's due to the fact that we currently build
## query_compile_time_config only in Mbed TLS repo and not in the TF-PSA-Crypto
## one. Once we'll have in both repos this check can be removed.
has_query_compile_time_config () {
if ! [ -f "$1/programs/test/query_compile_time_config" ]; then
return 1;
fi
return 0;
}
if has_query_compile_time_config "$root_dir"; then
query_compile_time_config_dir="$root_dir/programs/test"
elif is_valid_root "$root_dir/.." && has_query_compile_time_config "$root_dir/.."; then
query_compile_time_config_dir="$root_dir/../programs/test"
else
query_compile_time_config_dir=""
fi
## config_has SYMBOL...
## Succeeds if the library configuration has all SYMBOLs set.
##
## Note: depending on the above check query_compile_time_config_dir might be
## intentionally set to "". In this case the following function will fail.
config_has () {
for x in "$@"; do
# This function is commonly called in an if condition, where "set -e"
# has no effect, so make sure to stop explicitly on error.
"$query_compile_time_config_dir/query_compile_time_config" "$x" || return $?
done
}
## depends_on SYMBOL...
## Exit if the library configuration does not have all SYMBOLs set.
depends_on () {
if ! [ -f "$query_compile_time_config_dir/query_compile_time_config" ]; then
echo "query_compile_time_config is missing"
exit 127
fi
m=
for x in "$@"; do
if ! config_has "$x"; then
m="$m $x"
fi
done
if [ -n "$m" ]; then
cat >&2 <<EOF
$0: this demo requires the following
configuration options to be enabled at compile time:
$m
EOF
# Exit with a success status so that this counts as a pass for run_demos.py.
exit
fi
}
## Add the names of files to clean up to this whitespace-separated variable.
## The file names must not contain whitespace characters.
files_to_clean=
## Call this function at the end of each script.
## It is called automatically if the script is killed by a signal.
cleanup () {
rm -f -- $files_to_clean
}
################################################################
## End of the public interfaces. Code beyond this point is not
## meant to be called directly from a demo script.
trap 'cleanup; trap - HUP; kill -HUP $$' HUP
trap 'cleanup; trap - INT; kill -INT $$' INT
trap 'cleanup; trap - TERM; kill -TERM $$' TERM
if config_has MBEDTLS_ENTROPY_NV_SEED; then
# Create a seedfile that's sufficiently long in all library configurations.
# This is necessary for programs that use randomness.
# Assume that the name of the seedfile is the default name.
files_to_clean="$files_to_clean seedfile"
dd if=/dev/urandom of=seedfile ibs=64 obs=64 count=1
fi

37
vendor/mbedtls/framework/scripts/doxygen.sh vendored Executable file
View File

@@ -0,0 +1,37 @@
#!/bin/sh
# Make sure the doxygen documentation builds without warnings
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
# Abort on errors (and uninitialised variables)
set -eu
. $(dirname "$0")/project_detection.sh
if in_mbedtls_repo || in_tf_psa_crypto_repo; then :; else
echo "Must be run from Mbed TLS root or TF-PSA-Crypto root" >&2
exit 1
fi
if $(dirname "$0")/apidoc_full.sh > doc.out 2>doc.err; then :; else
cat doc.err
echo "FAIL" >&2
exit 1;
fi
cat doc.out doc.err | \
grep -v "warning: ignoring unsupported tag" \
> doc.filtered
if grep -E "(warning|error):" doc.filtered; then
echo "FAIL" >&2
exit 1;
fi
if in_mbedtls_repo && in_3_6_branch; then
make apidoc_clean
fi
rm -f doc.out doc.err doc.filtered

View File

@@ -0,0 +1,237 @@
#!/usr/bin/env python3
"""
Purpose
This script dumps comb table of ec curve. When you add a new ec curve, you
can use this script to generate codes to define `<curve>_T` in ecp_curves.c
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import os
import subprocess
import sys
import tempfile
HOW_TO_ADD_NEW_CURVE = """
If you are trying to add new curve, you can follow these steps:
1. Define curve parameters (<curve>_p, <curve>_gx, etc...) in ecp_curves.c.
2. Add a macro to define <curve>_T to NULL following these parameters.
3. Build mbedcrypto
4. Run this script with an argument of new curve
5. Copy the output of this script into ecp_curves.c and replace the macro added
in Step 2
6. Rebuild and test if everything is ok
Replace the <curve> in the above with the name of the curve you want to add."""
CC = os.getenv('CC', 'cc')
MBEDTLS_LIBRARY_PATH = os.getenv('MBEDTLS_LIBRARY_PATH', "library")
SRC_DUMP_COMB_TABLE = r'''
#include <stdio.h>
#include <stdlib.h>
#include "mbedtls/ecp.h"
#include "mbedtls/error.h"
static void dump_mpi_initialize( const char *name, const mbedtls_mpi *d )
{
uint8_t buf[128] = {0};
size_t olen;
uint8_t *p;
olen = mbedtls_mpi_size( d );
mbedtls_mpi_write_binary_le( d, buf, olen );
printf("static const mbedtls_mpi_uint %s[] = {\n", name);
for (p = buf; p < buf + olen; p += 8) {
printf( " BYTES_TO_T_UINT_8( 0x%02X, 0x%02X, 0x%02X, 0x%02X, 0x%02X, 0x%02X, 0x%02X, 0x%02X ),\n",
p[0], p[1], p[2], p[3], p[4], p[5], p[6], p[7] );
}
printf("};\n");
}
static void dump_T( const mbedtls_ecp_group *grp )
{
char name[128];
printf( "#if MBEDTLS_ECP_FIXED_POINT_OPTIM == 1\n" );
for (size_t i = 0; i < grp->T_size; ++i) {
snprintf( name, sizeof(name), "%s_T_%zu_X", CURVE_NAME, i );
dump_mpi_initialize( name, &grp->T[i].X );
snprintf( name, sizeof(name), "%s_T_%zu_Y", CURVE_NAME, i );
dump_mpi_initialize( name, &grp->T[i].Y );
}
printf( "static const mbedtls_ecp_point %s_T[%zu] = {\n", CURVE_NAME, grp->T_size );
size_t olen;
for (size_t i = 0; i < grp->T_size; ++i) {
int z;
if ( mbedtls_mpi_cmp_int(&grp->T[i].Z, 0) == 0 ) {
z = 0;
} else if ( mbedtls_mpi_cmp_int(&grp->T[i].Z, 1) == 0 ) {
z = 1;
} else {
fprintf( stderr, "Unexpected value of Z (i = %d)\n", (int)i );
exit( 1 );
}
printf( " ECP_POINT_INIT_XY_Z%d(%s_T_%zu_X, %s_T_%zu_Y),\n",
z,
CURVE_NAME, i,
CURVE_NAME, i
);
}
printf("};\n#endif\n\n");
}
int main()
{
int rc;
mbedtls_mpi m;
mbedtls_ecp_point R;
mbedtls_ecp_group grp;
mbedtls_ecp_group_init( &grp );
rc = mbedtls_ecp_group_load( &grp, CURVE_ID );
if (rc != 0) {
char buf[100];
mbedtls_strerror( rc, buf, sizeof(buf) );
fprintf( stderr, "mbedtls_ecp_group_load: %s (-0x%x)\n", buf, -rc );
return 1;
}
grp.T = NULL;
mbedtls_ecp_point_init( &R );
mbedtls_mpi_init( &m);
mbedtls_mpi_lset( &m, 1 );
rc = mbedtls_ecp_mul( &grp, &R, &m, &grp.G, NULL, NULL );
if ( rc != 0 ) {
char buf[100];
mbedtls_strerror( rc, buf, sizeof(buf) );
fprintf( stderr, "mbedtls_ecp_mul: %s (-0x%x)\n", buf, -rc );
return 1;
}
if ( grp.T == NULL ) {
fprintf( stderr, "grp.T is not generated. Please make sure"
"MBEDTLS_ECP_FIXED_POINT_OPTIM is enabled in mbedtls_config.h\n" );
return 1;
}
dump_T( &grp );
return 0;
}
'''
SRC_DUMP_KNOWN_CURVE = r'''
#include <stdio.h>
#include <stdlib.h>
#include "mbedtls/ecp.h"
int main() {
const mbedtls_ecp_curve_info *info = mbedtls_ecp_curve_list();
mbedtls_ecp_group grp;
mbedtls_ecp_group_init( &grp );
while ( info->name != NULL ) {
mbedtls_ecp_group_load( &grp, info->grp_id );
if ( mbedtls_ecp_get_type(&grp) == MBEDTLS_ECP_TYPE_SHORT_WEIERSTRASS ) {
printf( " %s", info->name );
}
info++;
}
printf( "\n" );
return 0;
}
'''
def join_src_path(*args):
return os.path.normpath(os.path.join(os.path.dirname(__file__), "..", *args))
def run_c_source(src, cflags):
"""
Compile and run C source code
:param src: the c language code to run
:param cflags: additional cflags passing to compiler
:return:
"""
binname = tempfile.mktemp(prefix="mbedtls")
fd, srcname = tempfile.mkstemp(prefix="mbedtls", suffix=".c")
srcfile = os.fdopen(fd, mode="w")
srcfile.write(src)
srcfile.close()
args = [CC,
*cflags,
'-I' + join_src_path("include"),
"-o", binname,
'-L' + MBEDTLS_LIBRARY_PATH,
srcname,
'-lmbedcrypto']
p = subprocess.run(args=args, check=False)
if p.returncode != 0:
return False
p = subprocess.run(args=[binname], check=False, env={
'LD_LIBRARY_PATH': MBEDTLS_LIBRARY_PATH
})
if p.returncode != 0:
return False
os.unlink(srcname)
os.unlink(binname)
return True
def compute_curve(curve):
"""compute comb table for curve"""
r = run_c_source(
SRC_DUMP_COMB_TABLE,
[
'-g',
'-DCURVE_ID=MBEDTLS_ECP_DP_%s' % curve.upper(),
'-DCURVE_NAME="%s"' % curve.lower(),
])
if not r:
print("""\
Unable to compile and run utility.""", file=sys.stderr)
sys.exit(1)
def usage():
print("""
Usage: python %s <curve>...
Arguments:
curve Specify one or more curve names (e.g secp256r1)
All possible curves: """ % sys.argv[0])
run_c_source(SRC_DUMP_KNOWN_CURVE, [])
print("""
Environment Variable:
CC Specify which c compile to use to compile utility.
MBEDTLS_LIBRARY_PATH
Specify the path to mbedcrypto library. (e.g. build/library/)
How to add a new curve: %s""" % HOW_TO_ADD_NEW_CURVE)
def run_main():
shared_lib_path = os.path.normpath(os.path.join(MBEDTLS_LIBRARY_PATH, "libmbedcrypto.so"))
static_lib_path = os.path.normpath(os.path.join(MBEDTLS_LIBRARY_PATH, "libmbedcrypto.a"))
if not os.path.exists(shared_lib_path) and not os.path.exists(static_lib_path):
print("Warning: both '%s' and '%s' are not exists. This script will use "
"the library from your system instead of the library compiled by "
"this source directory.\n"
"You can specify library path using environment variable "
"'MBEDTLS_LIBRARY_PATH'." % (shared_lib_path, static_lib_path),
file=sys.stderr)
if len(sys.argv) <= 1:
usage()
else:
for curve in sys.argv[1:]:
compute_curve(curve)
if __name__ == '__main__':
run_main()

View File

@@ -0,0 +1,96 @@
#!/usr/bin/env perl
#
# Based on NIST CTR_DRBG.rsp validation file
# Only uses AES-256-CTR cases that use a Derivation function
# and concats nonce and personalization for initialization.
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use strict;
my $file = shift;
open(TEST_DATA, "$file") or die "Opening test cases '$file': $!";
sub get_suite_val($)
{
my $name = shift;
my $val = "";
my $line = <TEST_DATA>;
($val) = ($line =~ /\[$name\s\=\s(\w+)\]/);
return $val;
}
sub get_val($)
{
my $name = shift;
my $val = "";
my $line;
while($line = <TEST_DATA>)
{
next if($line !~ /=/);
last;
}
($val) = ($line =~ /^$name = (\w+)/);
return $val;
}
my $cnt = 1;;
while (my $line = <TEST_DATA>)
{
next if ($line !~ /^\[AES-256 use df/);
my $PredictionResistanceStr = get_suite_val("PredictionResistance");
my $PredictionResistance = 0;
$PredictionResistance = 1 if ($PredictionResistanceStr eq 'True');
my $EntropyInputLen = get_suite_val("EntropyInputLen");
my $NonceLen = get_suite_val("NonceLen");
my $PersonalizationStringLen = get_suite_val("PersonalizationStringLen");
my $AdditionalInputLen = get_suite_val("AdditionalInputLen");
for ($cnt = 0; $cnt < 15; $cnt++)
{
my $Count = get_val("COUNT");
my $EntropyInput = get_val("EntropyInput");
my $Nonce = get_val("Nonce");
my $PersonalizationString = get_val("PersonalizationString");
my $AdditionalInput1 = get_val("AdditionalInput");
my $EntropyInputPR1 = get_val("EntropyInputPR") if ($PredictionResistance == 1);
my $EntropyInputReseed = get_val("EntropyInputReseed") if ($PredictionResistance == 0);
my $AdditionalInputReseed = get_val("AdditionalInputReseed") if ($PredictionResistance == 0);
my $AdditionalInput2 = get_val("AdditionalInput");
my $EntropyInputPR2 = get_val("EntropyInputPR") if ($PredictionResistance == 1);
my $ReturnedBits = get_val("ReturnedBits");
if ($PredictionResistance == 1)
{
print("CTR_DRBG NIST Validation (AES-256 use df,$PredictionResistanceStr,$EntropyInputLen,$NonceLen,$PersonalizationStringLen,$AdditionalInputLen) #$Count\n");
print("ctr_drbg_validate_pr");
print(":\"$Nonce$PersonalizationString\"");
print(":\"$EntropyInput$EntropyInputPR1$EntropyInputPR2\"");
print(":\"$AdditionalInput1\"");
print(":\"$AdditionalInput2\"");
print(":\"$ReturnedBits\"");
print("\n\n");
}
else
{
print("CTR_DRBG NIST Validation (AES-256 use df,$PredictionResistanceStr,$EntropyInputLen,$NonceLen,$PersonalizationStringLen,$AdditionalInputLen) #$Count\n");
print("ctr_drbg_validate_nopr");
print(":\"$Nonce$PersonalizationString\"");
print(":\"$EntropyInput$EntropyInputReseed\"");
print(":\"$AdditionalInput1\"");
print(":\"$AdditionalInputReseed\"");
print(":\"$AdditionalInput2\"");
print(":\"$ReturnedBits\"");
print("\n\n");
}
}
}
close(TEST_DATA);

View File

@@ -0,0 +1,101 @@
#!/usr/bin/env perl
#
# Based on NIST gcmDecryptxxx.rsp validation files
# Only first 3 of every set used for compile time saving
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use strict;
my $file = shift;
open(TEST_DATA, "$file") or die "Opening test cases '$file': $!";
sub get_suite_val($)
{
my $name = shift;
my $val = "";
while(my $line = <TEST_DATA>)
{
next if ($line !~ /^\[/);
($val) = ($line =~ /\[$name\s\=\s(\w+)\]/);
last;
}
return $val;
}
sub get_val($)
{
my $name = shift;
my $val = "";
my $line;
while($line = <TEST_DATA>)
{
next if($line !~ /=/);
last;
}
($val) = ($line =~ /^$name = (\w+)/);
return $val;
}
sub get_val_or_fail($)
{
my $name = shift;
my $val = "FAIL";
my $line;
while($line = <TEST_DATA>)
{
next if($line !~ /=/ && $line !~ /FAIL/);
last;
}
($val) = ($line =~ /^$name = (\w+)/) if ($line =~ /=/);
return $val;
}
my $cnt = 1;;
while (my $line = <TEST_DATA>)
{
my $key_len = get_suite_val("Keylen");
next if ($key_len !~ /\d+/);
my $iv_len = get_suite_val("IVlen");
my $pt_len = get_suite_val("PTlen");
my $add_len = get_suite_val("AADlen");
my $tag_len = get_suite_val("Taglen");
for ($cnt = 0; $cnt < 3; $cnt++)
{
my $Count = get_val("Count");
my $key = get_val("Key");
my $iv = get_val("IV");
my $ct = get_val("CT");
my $add = get_val("AAD");
my $tag = get_val("Tag");
my $pt = get_val_or_fail("PT");
print("GCM NIST Validation (AES-$key_len,$iv_len,$pt_len,$add_len,$tag_len) #$Count\n");
print("gcm_decrypt_and_verify");
print(":\"$key\"");
print(":\"$ct\"");
print(":\"$iv\"");
print(":\"$add\"");
print(":$tag_len");
print(":\"$tag\"");
print(":\"$pt\"");
print(":0");
print("\n\n");
}
}
print("GCM Selftest\n");
print("gcm_selftest:\n\n");
close(TEST_DATA);

View File

@@ -0,0 +1,84 @@
#!/usr/bin/env perl
#
# Based on NIST gcmEncryptIntIVxxx.rsp validation files
# Only first 3 of every set used for compile time saving
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use strict;
my $file = shift;
open(TEST_DATA, "$file") or die "Opening test cases '$file': $!";
sub get_suite_val($)
{
my $name = shift;
my $val = "";
while(my $line = <TEST_DATA>)
{
next if ($line !~ /^\[/);
($val) = ($line =~ /\[$name\s\=\s(\w+)\]/);
last;
}
return $val;
}
sub get_val($)
{
my $name = shift;
my $val = "";
my $line;
while($line = <TEST_DATA>)
{
next if($line !~ /=/);
last;
}
($val) = ($line =~ /^$name = (\w+)/);
return $val;
}
my $cnt = 1;;
while (my $line = <TEST_DATA>)
{
my $key_len = get_suite_val("Keylen");
next if ($key_len !~ /\d+/);
my $iv_len = get_suite_val("IVlen");
my $pt_len = get_suite_val("PTlen");
my $add_len = get_suite_val("AADlen");
my $tag_len = get_suite_val("Taglen");
for ($cnt = 0; $cnt < 3; $cnt++)
{
my $Count = get_val("Count");
my $key = get_val("Key");
my $pt = get_val("PT");
my $add = get_val("AAD");
my $iv = get_val("IV");
my $ct = get_val("CT");
my $tag = get_val("Tag");
print("GCM NIST Validation (AES-$key_len,$iv_len,$pt_len,$add_len,$tag_len) #$Count\n");
print("gcm_encrypt_and_tag");
print(":\"$key\"");
print(":\"$pt\"");
print(":\"$iv\"");
print(":\"$add\"");
print(":\"$ct\"");
print(":$tag_len");
print(":\"$tag\"");
print(":0");
print("\n\n");
}
}
print("GCM Selftest\n");
print("gcm_selftest:\n\n");
close(TEST_DATA);

View File

@@ -0,0 +1,74 @@
#!/usr/bin/env perl
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use strict;
my $file = shift;
open(TEST_DATA, "$file") or die "Opening test cases '$file': $!";
sub get_val($$)
{
my $str = shift;
my $name = shift;
my $val = "";
while(my $line = <TEST_DATA>)
{
next if($line !~ /^# $str/);
last;
}
while(my $line = <TEST_DATA>)
{
last if($line eq "\r\n");
$val .= $line;
}
$val =~ s/[ \r\n]//g;
return $val;
}
my $state = 0;
my $val_n = "";
my $val_e = "";
my $val_p = "";
my $val_q = "";
my $mod = 0;
my $cnt = 1;
while (my $line = <TEST_DATA>)
{
next if ($line !~ /^# Example/);
( $mod ) = ($line =~ /A (\d+)/);
$val_n = get_val("RSA modulus n", "N");
$val_e = get_val("RSA public exponent e", "E");
$val_p = get_val("Prime p", "P");
$val_q = get_val("Prime q", "Q");
for(my $i = 1; $i <= 6; $i++)
{
my $val_m = get_val("Message to be", "M");
my $val_salt = get_val("Salt", "Salt");
my $val_sig = get_val("Signature", "Sig");
print("RSASSA-PSS Signature Example ${cnt}_${i}\n");
print("pkcs1_rsassa_pss_sign:$mod:16:\"$val_p\":16:\"$val_q\":16:\"$val_n\":16:\"$val_e\":SIG_RSA_SHA1:MBEDTLS_MD_SHA1");
print(":\"$val_m\"");
print(":\"$val_salt\"");
print(":\"$val_sig\":0");
print("\n\n");
print("RSASSA-PSS Signature Example ${cnt}_${i} (verify)\n");
print("pkcs1_rsassa_pss_verify:$mod:16:\"$val_n\":16:\"$val_e\":SIG_RSA_SHA1:MBEDTLS_MD_SHA1");
print(":\"$val_m\"");
print(":\"$val_salt\"");
print(":\"$val_sig\":0");
print("\n\n");
}
$cnt++;
}
close(TEST_DATA);

View File

@@ -0,0 +1,71 @@
#!/bin/sh
# This script splits the data test files containing the test cases into
# individual files (one test case per file) suitable for use with afl
# (American Fuzzy Lop). http://lcamtuf.coredump.cx/afl/
#
# Usage: generate-afl-tests.sh <test data file path>
# <test data file path> - should be the path to one of the test suite files
# such as 'test_suite_rsa.data'
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
# Abort on errors
set -e
if [ -z $1 ]
then
echo " [!] No test file specified" >&2
echo "Usage: $0 <test data file>" >&2
exit 1
fi
SRC_FILEPATH=$(dirname $1)/$(basename $1)
TESTSUITE=$(basename $1 .data)
THIS_DIR=$(basename $PWD)
if [ -d ../library -a -d ../include -a -d ../tests -a $THIS_DIR == "tests" ];
then :;
else
echo " [!] Must be run from Mbed TLS tests directory" >&2
exit 1
fi
DEST_TESTCASE_DIR=$TESTSUITE-afl-tests
DEST_OUTPUT_DIR=$TESTSUITE-afl-out
echo " [+] Creating output directories" >&2
if [ -e $DEST_OUTPUT_DIR/* ];
then :
echo " [!] Test output files already exist." >&2
exit 1
else
mkdir -p $DEST_OUTPUT_DIR
fi
if [ -e $DEST_TESTCASE_DIR/* ];
then :
echo " [!] Test output files already exist." >&2
else
mkdir -p $DEST_TESTCASE_DIR
fi
echo " [+] Creating test cases" >&2
cd $DEST_TESTCASE_DIR
split -p '^\s*$' ../$SRC_FILEPATH
for f in *;
do
# Strip out any blank lines (no trim on OS X)
sed '/^\s*$/d' $f >testcase_$f
rm $f
done
cd ..
echo " [+] Test cases in $DEST_TESTCASE_DIR" >&2

View File

@@ -0,0 +1,434 @@
#!/usr/bin/env python3
"""Generate test data for bignum functions.
With no arguments, generate all test data. With non-option arguments,
generate only the specified files.
Class structure:
Child classes of test_data_generation.BaseTarget (file targets) represent an output
file. These indicate where test cases will be written to, for all subclasses of
this target. Multiple file targets should not reuse a `target_basename`.
Each subclass derived from a file target can either be:
- A concrete class, representing a test function, which generates test cases.
- An abstract class containing shared methods and attributes, not associated
with a test function. An example is BignumOperation, which provides
common features used for bignum binary operations.
Both concrete and abstract subclasses can be derived from, to implement
additional test cases (see BignumCmp and BignumCmpAbs for examples of deriving
from abstract and concrete classes).
Adding test case generation for a function:
A subclass representing the test function should be added, deriving from a
file target such as BignumTarget. This test class must set/implement the
following:
- test_function: the function name from the associated .function file.
- test_name: a descriptive name or brief summary to refer to the test
function.
- arguments(): a method to generate the list of arguments required for the
test_function.
- generate_function_tests(): a method to generate TestCases for the function.
This should create instances of the class with required input data, and
call `.create_test_case()` to yield the TestCase.
Additional details and other attributes/methods are given in the documentation
of BaseTarget in test_data_generation.py.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import sys
import math
from abc import ABCMeta
from typing import List
from mbedtls_framework import test_data_generation
from mbedtls_framework import bignum_common
# Import modules containing additional test classes
# Test function classes in these modules will be registered by
# the framework
from mbedtls_framework import bignum_core, bignum_mod_raw, bignum_mod # pylint: disable=unused-import
class BignumTarget(test_data_generation.BaseTarget):
#pylint: disable=too-few-public-methods
"""Target for bignum (legacy) test case generation."""
target_basename = 'test_suite_bignum.generated'
class BignumOperation(bignum_common.OperationCommon, BignumTarget,
metaclass=ABCMeta):
#pylint: disable=abstract-method
"""Common features for bignum operations in legacy tests."""
unique_combinations_only = True
input_values = [
"", "0", "-", "-0",
"7b", "-7b",
"0000000000000000123", "-0000000000000000123",
"1230000000000000000", "-1230000000000000000"
]
def description_suffix(self) -> str:
#pylint: disable=no-self-use # derived classes need self
"""Text to add at the end of the test case description."""
return ""
def description(self) -> str:
"""Generate a description for the test case.
If not set, case_description uses the form A `symbol` B, where symbol
is used to represent the operation. Descriptions of each value are
generated to provide some context to the test case.
"""
if not self.case_description:
self.case_description = "{} {} {}".format(
self.value_description(self.arg_a),
self.symbol,
self.value_description(self.arg_b)
)
description_suffix = self.description_suffix()
if description_suffix:
self.case_description += " " + description_suffix
return super().description()
@staticmethod
def value_description(val) -> str:
"""Generate a description of the argument val.
This produces a simple description of the value, which is used in test
case naming to add context.
"""
if val == "":
return "0 (null)"
if val == "-":
return "negative 0 (null)"
if val == "0":
return "0 (1 limb)"
if val[0] == "-":
tmp = "negative"
val = val[1:]
else:
tmp = "positive"
if val[0] == "0":
tmp += " with leading zero limb"
elif len(val) > 10:
tmp = "large " + tmp
return tmp
class BignumGCDInvModOperation(BignumOperation):
#pylint: disable=abstract-method
"""Common features for testing GCD and Invmod functions."""
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a=val_a, val_b=val_b)
def description_suffix(self) -> str:
comparison_symbol = '='
if abs(self.int_a) > abs(self.int_b):
comparison_symbol = '>'
elif abs(self.int_a) < abs(self.int_b):
comparison_symbol = '<'
suffix_parts = [
f"|A|{comparison_symbol}|N|",
*(["A<0"] if self.int_a < 0 else []),
*(["N<0"] if self.int_b < 0 else []),
"A=0" if self.int_a == 0 else f"A {'even' if self.int_a % 2 == 0 else 'odd'}",
"N=0" if self.int_b == 0 else f"B {'even' if self.int_b % 2 == 0 else 'odd'}"
]
return ": " + ", ".join(suffix_parts)
# The default values from BignumOperation are not useful, so overwrite them.
input_values = [
"c79e27fc71c69a08b3e85bd48b9cd3be9aa8e2e56df39f4ed8",
"299dd34be98436729eb10f690f8d2bfc5bee21984b775e1e75",
"-ecbb3a4e986d488172ecd54f7bd71bd18050c4ed",
"7da9ec44f42e6311c56a",
"cdbcce3f763819345cfb",
"100000000", "300000000", "500000000",
"50000", "30000",
"1", "2", "3", "", "00", "-1"
]
input_cases = [
("bc7fa9fb389618302e8b", "d49730e586607d42269f"),
("28bcc01a2d54b174532e", "d1915057d829a934c25d"),
("d56b50834719280dfa1d", "f007b78f6278ebcccd57"),
("8c327d1d8743c89d4483", "aa20b0c1f97a428311b5"),
("e905382f38", "c844b4f9bdaa5ed0002df3dbd2991cd9b9d"),
("e4623ef13d", "f2a4894ede013e354e481fe8974e67"),
("9f6afa8bdb", "b50aa03a7066df6f27bd6267b"),
("95f99b7122", "e8c74031ec75839f7539"),
("32", "948fbec067"),
("7445", "948fbec067"),
("31850e", "948fbec067"),
("421c2cc8", "948fbec067"),
("32a69", "71e107"),
("36d4e9", "3e05d1"),
("babf01", "1bf699d1"),
("7", "31"),
]
def get_return_code_gcd_modinv_odd_gcd_only(self) -> str:
code = "0"
if (self.int_a > self.int_b) or \
(self.int_a < 0) or \
(self.int_b % 2 == 0):
code = "MBEDTLS_ERR_MPI_BAD_INPUT_DATA"
return code
def get_return_code_gcd_modinv_odd(self) -> str:
if self.int_b < 2:
return "MBEDTLS_ERR_MPI_BAD_INPUT_DATA"
return self.get_return_code_gcd_modinv_odd_gcd_only()
class BignumCmp(BignumOperation):
"""Test cases for bignum value comparison."""
count = 0
test_function = "mpi_cmp_mpi"
test_name = "MPI compare"
input_cases = [
("-2", "-3"),
("-2", "-2"),
("2b4", "2b5"),
("2b5", "2b6")
]
def __init__(self, val_a, val_b) -> None:
super().__init__(val_a, val_b)
self._result = int(self.int_a > self.int_b) - int(self.int_a < self.int_b)
self.symbol = ["<", "==", ">"][self._result + 1]
def result(self) -> List[str]:
return [str(self._result)]
class BignumCmpAbs(BignumCmp):
"""Test cases for absolute bignum value comparison."""
count = 0
test_function = "mpi_cmp_abs"
test_name = "MPI compare (abs)"
def __init__(self, val_a, val_b) -> None:
super().__init__(val_a.strip("-"), val_b.strip("-"))
class BignumInvMod(BignumOperation):
"""Test cases for bignum modular inverse."""
count = 0
symbol = "^-1 mod"
test_function = "mpi_inv_mod"
test_name = "MPI inv_mod"
# The default values are not very useful here, so clear them.
input_values = [] # type: List[str]
input_cases = bignum_common.combination_two_lists(
# Input values for A
bignum_common.expand_list_negative([
"aa4df5cb14b4c31237f98bd1faf527c283c2d0f3eec89718664ba33f9762907c",
"f847e7731a2687c837f6b825f2937d997bf66814d3db79b27b",
"2ec0888f",
"22fbdf4c",
"32cf9a75",
]),
# Input values for N - must be positive.
[
"fffbbd660b94412ae61ead9c2906a344116e316a256fd387874c6c675b1d587d",
"2fe72fa5c05bc14c1279e37e2701bd956822999f42c5cbe84",
"2ec0888f",
"22fbdf4c",
"34d0830",
"364b6729",
"14419cd",
],
)
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a, val_b)
if math.gcd(self.int_a, self.int_b) == 1:
self._result = bignum_common.invmod_positive(self.int_a, self.int_b)
else:
self._result = -1 # No modular inverse.
def description_suffix(self) -> str:
suffix = ": "
# Assuming N (int_b) is always positive, compare absolute values,
# but only print the absolute value bars when A is negative.
a_str = "A" if (self.int_a >= 0) else "|A|"
if abs(self.int_a) > self.int_b:
suffix += f"{a_str}>N"
elif abs(self.int_a) < self.int_b:
suffix += f"{a_str}<N"
else:
suffix += f"{a_str}=N"
if self.int_a < 0:
suffix += ", A<0"
if self._result == -1:
suffix += ", no inverse"
return suffix
def result(self) -> List[str]:
if self._result == -1: # No modular inverse.
return [bignum_common.quote_str("0"), "MBEDTLS_ERR_MPI_NOT_ACCEPTABLE"]
return [bignum_common.quote_str("{:x}".format(self._result)), "0"]
class BignumGCD(BignumOperation):
"""Test cases for greatest common divisor."""
count = 0
symbol = "GCD"
test_function = "mpi_gcd"
test_name = "GCD"
# The default values are not very useful here, so overwrite them.
input_values = bignum_common.expand_list_negative([
"3c094fd6b36ee4902c8ba84d13a401def90a2130116dad3361",
"b2b06ebe14a185a83d5d2d7bddd1dd0e05e800d6b914fbed4e",
"203265b387",
"9bc8e63852",
"100000000",
"300000000",
"500000000",
"50000",
"30000",
"1",
"2",
"3",
])
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a, val_b)
# We always expect a positive result as the test data
# does not contain zero.
self._result = math.gcd(self.int_a, self.int_b)
def description_suffix(self) -> str:
suffix = ": "
if abs(self.int_a) > abs(self.int_b):
suffix += "|A|>|B|"
elif abs(self.int_a) < abs(self.int_b):
suffix += "|A|<|B|"
else:
suffix += "|A|=|B|"
if self.int_a < 0:
suffix += ", A<0"
if self.int_b < 0:
suffix += ", B<0"
suffix += ", A even" if (self.int_a % 2 == 0) else ", A odd"
suffix += ", B even" if (self.int_b % 2 == 0) else ", B odd"
return suffix
def result(self) -> List[str]:
return [bignum_common.quote_str("{:x}".format(self._result))]
class BignumGCDModInvOdd(BignumGCDInvModOperation):
"""Test cases for both modular inverse and greatest common divisor."""
count = 0
symbol = "GCD & ^-1 mod"
test_function = "mpi_gcd_modinv_odd_both"
test_name = "GCD & mod inv"
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a, val_b)
self._result_code = self.get_return_code_gcd_modinv_odd()
self._result_gcd = math.gcd(self.int_a, self.int_b)
# Only compute the modular inverse if we will get a result - negative
# and zero Ns are also present in the test data so skip them too.
if self._result_gcd == 1 and self.int_b > 1:
self._result_invmod = \
bignum_common.invmod_positive(self.int_a, self.int_b) # type: int | None
else:
self._result_invmod = None # No inverse
def result(self) -> List[str]:
# The test requires us to tell it if there is no modular inverse.
if self._result_invmod is None:
result_invmod = "no_inverse"
else:
result_invmod = "{:x}".format(self._result_invmod)
return [
self.format_result(self._result_gcd),
bignum_common.quote_str(result_invmod),
self._result_code,
]
class BignumGCDModInvOddOnlyGCD(BignumGCDInvModOperation):
"""Test cases for greatest common divisor only."""
count = 0
symbol = "GCD"
test_function = "mpi_gcd_modinv_odd_only_gcd"
test_name = "GCD only"
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a, val_b)
self._result_code = self.get_return_code_gcd_modinv_odd_gcd_only()
# We always expect a positive result as the function should reject
# negative inputs.
self._result_gcd = math.gcd(self.int_a, self.int_b)
def result(self) -> List[str]:
return [self.format_result(self._result_gcd), self._result_code]
class BignumGCDModInvOddOnlyModInv(BignumGCDInvModOperation):
"""Test cases for modular inverse only."""
count = 0
symbol = "^-1 mod"
test_function = "mpi_gcd_modinv_odd_only_modinv"
test_name = "Mod inv only"
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a, val_b)
self._result_code = self.get_return_code_gcd_modinv_odd()
# Only compute the modular inverse if we will get a result - negative
# and zero Ns are also present in the test data so skip them too.
if math.gcd(self.int_a, self.int_b) == 1 and self.int_b > 1:
self._result_invmod = \
bignum_common.invmod_positive(self.int_a, self.int_b) # type: int | None
else:
self._result_invmod = None # No inverse
def result(self) -> List[str]:
# The test requires us to tell it if there is no modular inverse.
if self._result_invmod is None:
return [bignum_common.quote_str("no_inverse"), self._result_code]
return [self.format_result(self._result_invmod), self._result_code]
class BignumAdd(BignumOperation):
"""Test cases for bignum value addition."""
count = 0
symbol = "+"
test_function = "mpi_add_mpi"
test_name = "MPI add"
input_cases = bignum_common.combination_pairs(
[
"1c67967269c6", "9cde3",
"-1c67967269c6", "-9cde3",
]
)
def __init__(self, val_a: str, val_b: str) -> None:
super().__init__(val_a, val_b)
self._result = self.int_a + self.int_b
def description_suffix(self) -> str:
if (self.int_a >= 0 and self.int_b >= 0):
return "" # obviously positive result or 0
if (self.int_a <= 0 and self.int_b <= 0):
return "" # obviously negative result or 0
# The sign of the result is not obvious, so indicate it
return ", result{}0".format('>' if self._result > 0 else
'<' if self._result < 0 else '=')
def result(self) -> List[str]:
return [bignum_common.quote_str("{:x}".format(self._result))]
if __name__ == '__main__':
# Use the section of the docstring relevant to the CLI as description
test_data_generation.main(sys.argv[1:], "\n".join(__doc__.splitlines()[:4]))

View File

@@ -0,0 +1,209 @@
#!/usr/bin/env python3
"""Generate test data for configuration reporting.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import inspect
import re
import sys
from typing import Iterable, Iterator, List, Optional, Tuple
import project_scripts # pylint: disable=unused-import
import config
from mbedtls_framework import build_tree
from mbedtls_framework import config_common
from mbedtls_framework import test_case
from mbedtls_framework import test_data_generation
def single_setting_case(setting: config_common.Setting, when_on: bool,
dependencies: List[str],
note: Optional[str]) -> test_case.TestCase:
"""Construct a test case for a boolean setting.
This test case passes if the setting and its dependencies are enabled,
and is skipped otherwise.
* setting: the setting to be tested.
* when_on: True to test with the setting enabled, or False to test
with the setting disabled.
* dependencies: extra dependencies for the test case.
* note: a note to add after the setting name in the test description.
This is generally a summary of dependencies, and is generally empty
if the given setting is only tested once.
"""
base = setting.name if when_on else '!' + setting.name
tc = test_case.TestCase()
tc.set_function('pass')
description_suffix = ' (' + note + ')' if note else ''
tc.set_description('Config: ' + base + description_suffix)
tc.set_dependencies([base] + dependencies)
return tc
PSA_WANT_KEY_TYPE_KEY_PAIR_RE = \
re.compile(r'(?P<prefix>PSA_WANT_KEY_TYPE_(?P<type>\w+)_KEY_PAIR_)(?P<operation>\w+)\Z')
# If foo is a setting that is only meaningful when bar is enabled, set
# SIMPLE_DEPENDENCIES[foo]=bar. More generally, bar can be a colon-separated
# list of settings, meaning that all the settings must be enabled. Each setting
# in bar can be prefixed with '!' to negate it. This is the same syntax as a
# depends_on directive in test data.
# See also `dependencies_of_settting`.
SIMPLE_DEPENDENCIES = {
'MBEDTLS_AESCE_C': 'MBEDTLS_AES_C',
'MBEDTLS_AESNI_C': 'MBEDTLS_AES_C',
'MBEDTLS_ERROR_STRERROR_DUMMY': '!MBEDTLS_ERROR_C',
'MBEDTLS_GENPRIME': 'MBEDTLS_RSA_C',
'MBEDTLS_NO_DEFAULT_ENTROPY_SOURCES': 'MBEDTLS_ENTROPY_C',
'MBEDTLS_PKCS1_V15': 'MBEDTLS_RSA_C',
'MBEDTLS_PKCS1_V21': 'MBEDTLS_RSA_C',
'MBEDTLS_PSA_CRYPTO_CLIENT': '!MBEDTLS_PSA_CRYPTO_C',
'MBEDTLS_PSA_INJECT_ENTROPY': 'MBEDTLS_PSA_CRYPTO_C',
'MBEDTLS_PSA_ASSUME_EXCLUSIVE_BUFFERS': 'MBEDTLS_PSA_CRYPTO_C',
}
if build_tree.is_mbedtls_3_6():
SIMPLE_DEPENDENCIES['MBEDTLS_NO_PLATFORM_ENTROPY'] = 'MBEDTLS_ENTROPY_C'
BUILTIN_MODULE_ENABLEMENT_MACROS = [
'MBEDTLS_AES_C', 'MBEDTLS_CAMELLIA_C', 'MBEDTLS_GCM_C',
'MBEDTLS_ECP_C',
'MBEDTLS_RSA_C',
'MBEDTLS_SHA256_C', 'MBEDTLS_SHA512_C',
]
def dependencies_of_setting(cfg: config_common.Config,
setting: config_common.Setting) -> Optional[str]:
"""Return dependencies without which a setting is not meaningful.
The dependencies of a setting express when a setting can be enabled and
is relevant. For example, if ``check_config.h`` errors out when
``defined(FOO) && !defined(BAR)``, then ``BAR`` is a dependency of ``FOO``.
If ``FOO`` has no effect when ``CORGE`` is disabled, then ``CORGE``
is a dependency of ``FOO``.
The return value can be a colon-separated list of settings, if the setting
is only meaningful when all of these settings are enabled. Each setting can
be negated by prefixing them with '!'. This is the same syntax as a
depends_on directive in test data.
"""
#pylint: disable=too-many-branches,too-many-return-statements
name = setting.name
if name in SIMPLE_DEPENDENCIES:
return SIMPLE_DEPENDENCIES[name]
if name.startswith('MBEDTLS_') and not name.endswith('_C'):
if name.startswith('MBEDTLS_CIPHER_PADDING_'):
return 'MBEDTLS_CIPHER_C:MBEDTLS_CIPHER_MODE_CBC'
if name.startswith('MBEDTLS_PK_PARSE_EC_'):
return 'MBEDTLS_PK_C:' + test_case.psa_or_3_6_feature_macro(
'PSA_KEY_TYPE_ECC_PUBLIC_KEY', test_case.Domain36.USE_PSA)
# For TLS settings, insist on having them once off and once on in
# a configuration where both client support and server support are
# enabled. The settings are also meaningful when only one side is
# enabled, but there isn't much point in having separate records
# for client-side and server-side, so we keep things simple.
# Requiring both sides to be enabled also means we know we'll run
# tests that only run Mbed TLS against itself, which only run in
# configurations with both sides enabled.
if name.startswith('MBEDTLS_SSL_TLS1_3_') or \
name == 'MBEDTLS_SSL_EARLY_DATA':
return 'MBEDTLS_SSL_CLI_C:MBEDTLS_SSL_SRV_C:MBEDTLS_SSL_PROTO_TLS1_3'
if name.startswith('MBEDTLS_SSL_DTLS_'):
return 'MBEDTLS_SSL_CLI_C:MBEDTLS_SSL_SRV_C:MBEDTLS_SSL_PROTO_DTLS'
if name.startswith('MBEDTLS_SSL_'):
return 'MBEDTLS_SSL_CLI_C:MBEDTLS_SSL_SRV_C'
if name == 'MBEDTLS_ECDH_VARIANT_EVEREST_ENABLED' and \
not build_tree.is_mbedtls_3_6():
# In 1.x the module ecdh.c is removed. This option remains, with its
# historical name for compatibility. It is still only relevant when
# the built-in implementation of ECDH is enabled, but this condition
# is no longer expressed as MBEDTLS_ECDH_C.
return 'MBEDTLS_PSA_BUILTIN_ALG_ECDH'
for pos in re.finditer(r'_', name):
super_name = name[:pos.start()] + '_C'
if cfg.known(super_name):
return super_name
# If super_name refers to a macro that still enables a
# cryptographic module, but is no longer exposed as a configuration
# option in 4.0/1.0, return it as a dependency.
if super_name in BUILTIN_MODULE_ENABLEMENT_MACROS:
return super_name
if name.startswith('PSA_WANT_'):
deps = 'MBEDTLS_PSA_CRYPTO_CLIENT'
m = PSA_WANT_KEY_TYPE_KEY_PAIR_RE.match(name)
if m and m.group('operation') != 'BASIC':
deps += ':' + m.group('prefix') + 'BASIC'
return deps
return None
def conditions_for_setting(cfg: config_common.Config,
setting: config_common.Setting
) -> Iterator[Tuple[List[str], str]]:
"""Enumerate the conditions under which to test the given setting.
* cfg: all configuration settings.
* setting: the setting to be tested.
Generate a stream of conditions, i.e. extra dependencies to test with
together with a human-readable explanation of each dependency. Some
typical cases:
* By default, generate a one-element stream with no extra dependencies.
* If the setting is ignored unless some other setting is enabled, generate
a one-element stream with that other setting as an extra dependency.
* If the setting is known to interact with some other setting, generate
a stream with one element where this setting is on and one where it's off.
* To skip the setting altogether, generate an empty stream.
"""
name = setting.name
if name.endswith('_ALT') and not config.is_seamless_alt(name):
# We don't test alt implementations, except (most) platform alts
return
dependencies = dependencies_of_setting(cfg, setting)
if dependencies:
yield [dependencies], ''
return
yield [], ''
def enumerate_boolean_setting_cases(cfg: config_common.Config
) -> Iterable[test_case.TestCase]:
"""Emit test cases for all boolean settings."""
for name in sorted(cfg.settings.keys()):
setting = cfg.settings[name]
if not name.startswith('PSA_WANT_') and setting.value:
continue # non-boolean setting
for when_on in True, False:
for deps, note in conditions_for_setting(cfg, setting):
yield single_setting_case(setting, when_on, deps, note)
class ConfigTestGenerator(test_data_generation.TestGenerator):
"""Generate test cases for configuration reporting."""
def __init__(self, settings):
# pylint: disable=no-member
config_members = dict(inspect.getmembers(config))
if 'MbedTLSConfig' in config_members:
self.mbedtls_config = config.MbedTLSConfig()
self.targets['test_suite_config.mbedtls_boolean'] = \
lambda: enumerate_boolean_setting_cases(self.mbedtls_config)
if 'CryptoConfig' in config_members:
if build_tree.is_mbedtls_3_6():
self.psa_config = config.CryptoConfig()
self.targets['test_suite_config.psa_boolean'] = \
lambda: enumerate_boolean_setting_cases(self.psa_config)
elif 'TFPSACryptoConfig' in config_members:
self.psa_config = config.TFPSACryptoConfig()
self.targets['test_suite_config.psa_boolean'] = \
lambda: enumerate_boolean_setting_cases(self.psa_config)
super().__init__(settings)
if __name__ == '__main__':
test_data_generation.main(sys.argv[1:], __doc__, ConfigTestGenerator)

View File

@@ -0,0 +1,21 @@
#!/usr/bin/env python3
"""Generate test data for ecp functions.
The command line usage, class structure and available methods are the same
as in generate_bignum_tests.py.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import sys
from mbedtls_framework import test_data_generation
# Import modules containing additional test classes
# Test function classes in these modules will be registered by
# the framework
from mbedtls_framework import ecp # pylint: disable=unused-import
if __name__ == '__main__':
# Use the section of the docstring relevant to the CLI as description
test_data_generation.main(sys.argv[1:], "\n".join(__doc__.splitlines()[:4]))

View File

@@ -0,0 +1,186 @@
#!/usr/bin/env python3
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
"""
Make fuzz like testing for pkcs7 tests
Given a valid DER pkcs7 file add tests to the test_suite_pkcs7.data file
- It is expected that the pkcs7_asn1_fail( data_t *pkcs7_buf )
function is defined in test_suite_pkcs7.function
- This is not meant to be portable code, if anything it is meant to serve as
documentation for showing how those ugly tests in test_suite_pkcs7.data were created
"""
import sys
from os.path import exists
from mbedtls_framework import test_case
PKCS7_TEST_FILE = "../suites/test_suite_pkcs7.data"
class Test: # pylint: disable=too-few-public-methods
"""
A instance of a test in test_suite_pkcs7.data
"""
def __init__(self, name, depends, func_call):
self.name = name
self.depends = depends
self.func_call = func_call
# pylint: disable=no-self-use
def to_string(self):
return "\n" + self.name + "\n" + self.depends + "\n" + self.func_call + "\n"
class TestData:
"""
Take in test_suite_pkcs7.data file.
Allow for new tests to be added.
"""
mandatory_dep = test_case.psa_or_3_6_feature_macro("PSA_ALG_SHA_256",
test_case.Domain36.USE_PSA)
test_name = "PKCS7 Parse Failure Invalid ASN1"
test_function = "pkcs7_asn1_fail:"
def __init__(self, file_name):
self.file_name = file_name
self.last_test_num, self.old_tests = self.read_test_file(file_name)
self.new_tests = []
# pylint: disable=no-self-use
def read_test_file(self, file):
"""
Parse the test_suite_pkcs7.data file.
"""
tests = []
if not exists(file):
print(file + " Does not exist")
sys.exit()
with open(file, "r", encoding='UTF-8') as fp:
data = fp.read()
lines = [line.strip() for line in data.split('\n') if len(line.strip()) > 1]
i = 0
while i < len(lines):
if "depends" in lines[i+1]:
tests.append(Test(lines[i], lines[i+1], lines[i+2]))
i += 3
else:
tests.append(Test(lines[i], None, lines[i+1]))
i += 2
latest_test_num = float(tests[-1].name.split('#')[1])
return latest_test_num, tests
def add(self, name, func_call):
self.last_test_num += 1
self.new_tests.append(Test(self.test_name + ": " + name + " #" + \
str(self.last_test_num), "depends_on:" + self.mandatory_dep, \
self.test_function + '"' + func_call + '"'))
def write_changes(self):
with open(self.file_name, 'a', encoding='UTF-8') as fw:
fw.write("\n")
for t in self.new_tests:
fw.write(t.to_string())
def asn1_mutate(data):
"""
We have been given an asn1 structure representing a pkcs7.
We want to return an array of slightly modified versions of this data
they should be modified in a way which makes the structure invalid
We know that asn1 structures are:
|---1 byte showing data type---|----byte(s) for length of data---|---data content--|
We know that some data types can contain other data types.
Return a dictionary of reasons and mutated data types.
"""
# off the bat just add bytes to start and end of the buffer
mutations = []
reasons = []
mutations.append(["00"] + data)
reasons.append("Add null byte to start")
mutations.append(data + ["00"])
reasons.append("Add null byte to end")
# for every asn1 entry we should attempt to:
# - change the data type tag
# - make the length longer than actual
# - make the length shorter than actual
i = 0
while i < len(data):
tag_i = i
leng_i = tag_i + 1
data_i = leng_i + 1 + (int(data[leng_i][1], 16) if data[leng_i][0] == '8' else 0)
if data[leng_i][0] == '8':
length = int(''.join(data[leng_i + 1: data_i]), 16)
else:
length = int(data[leng_i], 16)
tag = data[tag_i]
print("Looking at ans1: offset " + str(i) + " tag = " + tag + \
", length = " + str(length)+ ":")
print(''.join(data[data_i:data_i+length]))
# change tag to something else
if tag == "02":
# turn integers into octet strings
new_tag = "04"
else:
# turn everything else into an integer
new_tag = "02"
mutations.append(data[:tag_i] + [new_tag] + data[leng_i:])
reasons.append("Change tag " + tag + " to " + new_tag)
# change lengths to too big
# skip any edge cases which would cause carry over
if int(data[data_i - 1], 16) < 255:
new_length = str(hex(int(data[data_i - 1], 16) + 1))[2:]
if len(new_length) == 1:
new_length = "0"+new_length
mutations.append(data[:data_i -1] + [new_length] + data[data_i:])
reasons.append("Change length from " + str(length) + " to " \
+ str(length + 1))
# we can add another test here for tags that contain other tags \
# where they have more data than there containing tags account for
if tag in ["30", "a0", "31"]:
mutations.append(data[:data_i -1] + [new_length] + \
data[data_i:data_i + length] + ["00"] + \
data[data_i + length:])
reasons.append("Change contents of tag " + tag + " to contain \
one unaccounted extra byte")
# change lengths to too small
if int(data[data_i - 1], 16) > 0:
new_length = str(hex(int(data[data_i - 1], 16) - 1))[2:]
if len(new_length) == 1:
new_length = "0"+new_length
mutations.append(data[:data_i -1] + [new_length] + data[data_i:])
reasons.append("Change length from " + str(length) + " to " + str(length - 1))
# some tag types contain other tag types so we should iterate into the data
if tag in ["30", "a0", "31"]:
i = data_i
else:
i = data_i + length
return list(zip(reasons, mutations))
if __name__ == "__main__":
if len(sys.argv) < 2:
print("USAGE: " + sys.argv[0] + " <pkcs7_der_file>")
sys.exit()
DATA_FILE = sys.argv[1]
TEST_DATA = TestData(PKCS7_TEST_FILE)
with open(DATA_FILE, 'rb') as f:
DATA_STR = f.read().hex()
# make data an array of byte strings eg ['de','ad','be','ef']
HEX_DATA = list(map(''.join, [[DATA_STR[i], DATA_STR[i+1]] for i in range(0, len(DATA_STR), \
2)]))
# returns tuples of test_names and modified data buffers
MUT_ARR = asn1_mutate(HEX_DATA)
print("made " + str(len(MUT_ARR)) + " new tests")
for new_test in MUT_ARR:
TEST_DATA.add(new_test[0], ''.join(new_test[1]))
TEST_DATA.write_changes()

View File

@@ -0,0 +1,870 @@
#!/usr/bin/env python3
"""Generate test data for PSA cryptographic mechanisms.
With no arguments, generate all test data. With non-option arguments,
generate only the specified files.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import enum
import re
import sys
from typing import Callable, Dict, Iterable, Iterator, List, Optional
from mbedtls_framework import crypto_data_tests
from mbedtls_framework import crypto_knowledge
from mbedtls_framework import macro_collector #pylint: disable=unused-import
from mbedtls_framework import psa_information
from mbedtls_framework import psa_storage
from mbedtls_framework import psa_test_case
from mbedtls_framework import test_case
from mbedtls_framework import test_data_generation
def test_case_for_key_type_not_supported(
verb: str, key_type: str, bits: int,
not_supported_mechanism: str,
*args: str,
param_descr: str = ''
) -> test_case.TestCase:
"""Return one test case exercising a key creation method
for an unsupported key type or size.
"""
tc = psa_test_case.TestCase()
short_key_type = crypto_knowledge.short_expression(key_type)
tc.set_description('PSA {} {} {}-bit{} not supported'
.format(verb, short_key_type, bits,
' ' + param_descr if param_descr else ''))
# if tc.description == 'PSA import RSA_KEY_PAIR 1024-bit not supported':
# import pdb; pdb.set_trace()
tc.set_function(verb + '_not_supported')
tc.set_key_bits(bits)
tc.set_key_pair_usage([verb.upper()])
tc.assumes_not_supported(not_supported_mechanism)
tc.set_arguments([key_type] + list(args))
return tc
class KeyTypeNotSupported:
"""Generate test cases for when a key type is not supported."""
def __init__(self, info: psa_information.Information) -> None:
self.constructors = info.constructors
ALWAYS_SUPPORTED = frozenset([
'PSA_KEY_TYPE_DERIVE',
'PSA_KEY_TYPE_PASSWORD',
'PSA_KEY_TYPE_PASSWORD_HASH',
'PSA_KEY_TYPE_RAW_DATA',
'PSA_KEY_TYPE_HMAC'
])
def test_cases_for_key_type_not_supported(
self,
kt: crypto_knowledge.KeyType,
param: Optional[int] = None,
param_descr: str = '',
) -> Iterator[test_case.TestCase]:
"""Return test cases exercising key creation when the given type is unsupported.
If param is present and not None, emit test cases conditioned on this
parameter not being supported. If it is absent or None, emit test cases
conditioned on the base type not being supported.
"""
if kt.name in self.ALWAYS_SUPPORTED:
# Don't generate test cases for key types that are always supported.
# They would be skipped in all configurations, which is noise.
return
if param is None:
not_supported_mechanism = kt.name
else:
assert kt.params is not None
not_supported_mechanism = kt.params[param]
for bits in kt.sizes_to_test():
yield test_case_for_key_type_not_supported(
'import', kt.expression, bits,
not_supported_mechanism,
test_case.hex_string(kt.key_material(bits)),
param_descr=param_descr,
)
# Don't generate not-supported test cases for key generation of
# public keys. Our implementation always returns
# PSA_ERROR_INVALID_ARGUMENT when attempting to generate a
# public key, so we cover this together with the positive cases
# in the KeyGenerate class.
if not kt.is_public():
yield test_case_for_key_type_not_supported(
'generate', kt.expression, bits,
not_supported_mechanism,
str(bits),
param_descr=param_descr,
)
# To be added: derive
ECC_KEY_TYPES = ('PSA_KEY_TYPE_ECC_KEY_PAIR',
'PSA_KEY_TYPE_ECC_PUBLIC_KEY')
DH_KEY_TYPES = ('PSA_KEY_TYPE_DH_KEY_PAIR',
'PSA_KEY_TYPE_DH_PUBLIC_KEY')
def test_cases_for_not_supported(self) -> Iterator[test_case.TestCase]:
"""Generate test cases that exercise the creation of keys of unsupported types."""
for key_type in sorted(self.constructors.key_types):
if key_type in self.ECC_KEY_TYPES:
continue
if key_type in self.DH_KEY_TYPES:
continue
kt = crypto_knowledge.KeyType(key_type)
yield from self.test_cases_for_key_type_not_supported(kt)
for curve_family in sorted(self.constructors.ecc_curves):
for constr in self.ECC_KEY_TYPES:
kt = crypto_knowledge.KeyType(constr, [curve_family])
yield from self.test_cases_for_key_type_not_supported(
kt, param_descr='type')
yield from self.test_cases_for_key_type_not_supported(
kt, 0, param_descr='curve')
for dh_family in sorted(self.constructors.dh_groups):
for constr in self.DH_KEY_TYPES:
kt = crypto_knowledge.KeyType(constr, [dh_family])
yield from self.test_cases_for_key_type_not_supported(
kt, param_descr='type')
yield from self.test_cases_for_key_type_not_supported(
kt, 0, param_descr='group')
def test_case_for_key_generation(
key_type: str, bits: int,
*args: str,
result: str = ''
) -> test_case.TestCase:
"""Return one test case exercising a key generation.
"""
tc = psa_test_case.TestCase()
short_key_type = crypto_knowledge.short_expression(key_type)
tc.set_description('PSA {} {}-bit'
.format(short_key_type, bits))
tc.set_function('generate_key')
tc.set_key_bits(bits)
tc.set_key_pair_usage(['GENERATE'])
tc.set_arguments([key_type] + list(args) + [result])
return tc
class KeyGenerate:
"""Generate positive and negative (invalid argument) test cases for key generation."""
def __init__(self, info: psa_information.Information) -> None:
self.constructors = info.constructors
ECC_KEY_TYPES = ('PSA_KEY_TYPE_ECC_KEY_PAIR',
'PSA_KEY_TYPE_ECC_PUBLIC_KEY')
DH_KEY_TYPES = ('PSA_KEY_TYPE_DH_KEY_PAIR',
'PSA_KEY_TYPE_DH_PUBLIC_KEY')
@staticmethod
def test_cases_for_key_type_key_generation(
kt: crypto_knowledge.KeyType
) -> Iterator[test_case.TestCase]:
"""Return test cases exercising key generation.
All key types can be generated except for public keys. For public key
PSA_ERROR_INVALID_ARGUMENT status is expected.
"""
for bits in kt.sizes_to_test():
tc = test_case_for_key_generation(
kt.expression, bits,
str(bits),
'PSA_ERROR_INVALID_ARGUMENT' if kt.is_public() else 'PSA_SUCCESS'
)
if kt.is_public():
# The library checks whether the key type is a public key generically,
# before it reaches a point where it needs support for the specific key
# type, so it returns INVALID_ARGUMENT for unsupported public key types.
tc.set_dependencies([])
yield tc
def test_cases_for_key_generation(self) -> Iterator[test_case.TestCase]:
"""Generate test cases that exercise the generation of keys."""
for key_type in sorted(self.constructors.key_types):
if key_type in self.ECC_KEY_TYPES:
continue
if key_type in self.DH_KEY_TYPES:
continue
kt = crypto_knowledge.KeyType(key_type)
yield from self.test_cases_for_key_type_key_generation(kt)
for curve_family in sorted(self.constructors.ecc_curves):
for constr in self.ECC_KEY_TYPES:
kt = crypto_knowledge.KeyType(constr, [curve_family])
yield from self.test_cases_for_key_type_key_generation(kt)
for dh_family in sorted(self.constructors.dh_groups):
for constr in self.DH_KEY_TYPES:
kt = crypto_knowledge.KeyType(constr, [dh_family])
yield from self.test_cases_for_key_type_key_generation(kt)
class OpFail:
"""Generate test cases for operations that must fail."""
#pylint: disable=too-few-public-methods
class Reason(enum.Enum):
NOT_SUPPORTED = 0
INVALID = 1
INCOMPATIBLE = 2
PUBLIC = 3
def __init__(self, info: psa_information.Information) -> None:
self.constructors = info.constructors
key_type_expressions = self.constructors.generate_expressions(
sorted(self.constructors.key_types)
)
self.key_types = [crypto_knowledge.KeyType(kt_expr)
for kt_expr in key_type_expressions]
def make_test_case(
self,
alg: crypto_knowledge.Algorithm,
category: crypto_knowledge.AlgorithmCategory,
reason: 'Reason',
kt: Optional[crypto_knowledge.KeyType] = None,
not_supported: Optional[str] = None,
) -> test_case.TestCase:
"""Construct a failure test case for a one-key or keyless operation.
If `reason` is `Reason.NOT_SUPPORTED`, pass the not-supported
dependency symbol as the `not_supported` argument.
"""
#pylint: disable=too-many-arguments,too-many-locals
tc = psa_test_case.TestCase()
pretty_alg = alg.short_expression()
if reason == self.Reason.NOT_SUPPORTED:
assert not_supported is not None
pretty_reason = '!' + re.sub(r'PSA_WANT_[A-Z]+_', r'', not_supported)
else:
pretty_reason = reason.name.lower()
if kt:
key_type = kt.expression
pretty_type = kt.short_expression()
else:
key_type = ''
pretty_type = ''
tc.set_description('PSA {} {}: {}{}'
.format(category.name.lower(),
pretty_alg,
pretty_reason,
' with ' + pretty_type if pretty_type else ''))
tc.set_function(category.name.lower() + '_fail')
arguments = [] # type: List[str]
if kt:
bits = kt.sizes_to_test()[0]
if pretty_alg == "XTS" and kt.can_do(alg):
# XTS mode uses double-size keys for the underlying block cipher
bits = bits * 2
tc.set_key_bits(bits)
tc.set_key_pair_usage(['IMPORT'])
key_material = kt.key_material(bits)
arguments += [key_type, test_case.hex_string(key_material)]
arguments.append(alg.expression)
if category.is_asymmetric():
arguments.append('1' if reason == self.Reason.PUBLIC else '0')
error = ('NOT_SUPPORTED' if reason == self.Reason.NOT_SUPPORTED else
'INVALID_ARGUMENT')
arguments.append('PSA_ERROR_' + error)
if reason == self.Reason.NOT_SUPPORTED:
assert not_supported is not None
tc.assumes_not_supported(not_supported)
# Special case: if one of deterministic/randomized
# ECDSA is supported but not the other, then the one
# that is not supported in the signature direction is
# still supported in the verification direction,
# because the two verification algorithms are
# identical. This property is how Mbed TLS chooses to
# behave, the specification would also allow it to
# reject the algorithm. In the generated test cases,
# we avoid this difficulty by not running the
# not-supported test case when exactly one of the
# two variants is supported.
if not_supported == 'PSA_WANT_ALG_ECDSA':
tc.add_dependencies(['!PSA_WANT_ALG_DETERMINISTIC_ECDSA'])
if not_supported == 'PSA_WANT_ALG_DETERMINISTIC_ECDSA':
tc.add_dependencies(['!PSA_WANT_ALG_ECDSA'])
tc.set_arguments(arguments)
return tc
def no_key_test_cases(
self,
alg: crypto_knowledge.Algorithm,
category: crypto_knowledge.AlgorithmCategory,
) -> Iterator[test_case.TestCase]:
"""Generate failure test cases for keyless operations with the specified algorithm."""
if alg.can_do(category):
# Compatible operation, unsupported algorithm
for dep in psa_information.automatic_dependencies(alg.base_expression):
yield self.make_test_case(alg, category,
self.Reason.NOT_SUPPORTED,
not_supported=dep)
else:
# Incompatible operation, supported algorithm
yield self.make_test_case(alg, category, self.Reason.INVALID)
def one_key_test_cases(
self,
alg: crypto_knowledge.Algorithm,
category: crypto_knowledge.AlgorithmCategory,
) -> Iterator[test_case.TestCase]:
"""Generate failure test cases for one-key operations with the specified algorithm."""
for kt in self.key_types:
key_is_compatible = kt.can_do(alg)
if key_is_compatible and alg.can_do(category):
# Compatible key and operation, unsupported algorithm
for dep in psa_information.automatic_dependencies(alg.base_expression):
yield self.make_test_case(alg, category,
self.Reason.NOT_SUPPORTED,
kt=kt, not_supported=dep)
# Public key for a private-key operation
if category.is_asymmetric() and kt.is_public():
yield self.make_test_case(alg, category,
self.Reason.PUBLIC,
kt=kt)
elif key_is_compatible:
# Compatible key, incompatible operation, supported algorithm
yield self.make_test_case(alg, category,
self.Reason.INVALID,
kt=kt)
elif alg.can_do(category):
# Incompatible key, compatible operation, supported algorithm
yield self.make_test_case(alg, category,
self.Reason.INCOMPATIBLE,
kt=kt)
else:
# Incompatible key and operation. Don't test cases where
# multiple things are wrong, to keep the number of test
# cases reasonable.
pass
def test_cases_for_algorithm(
self,
alg: crypto_knowledge.Algorithm,
categories: Iterable[crypto_knowledge.AlgorithmCategory]
) -> Iterator[test_case.TestCase]:
"""Generate operation failure test cases for the specified algorithm."""
for category in categories:
if category.requires_key():
yield from self.one_key_test_cases(alg, category)
else:
yield from self.no_key_test_cases(alg, category)
def all_test_cases(self) -> Iterator[test_case.TestCase]:
"""Generate all test cases for operations that must fail."""
algorithm_constructors = sorted(self.constructors.algorithms)
algorithms = [crypto_knowledge.Algorithm(alg)
for alg in self.constructors.generate_expressions(
algorithm_constructors)]
supported_categories = set()
for alg in algorithms:
supported_categories.add(alg.category)
# We don't have a pake_fail test function yet.
# https://github.com/Mbed-TLS/mbedtls-framework/issues/263
supported_categories.remove(crypto_knowledge.AlgorithmCategory.PAKE)
categories = sorted(supported_categories, key=lambda cat: cat.value)
assert categories # sanity check: at least one category detected
for alg in algorithms:
yield from self.test_cases_for_algorithm(alg, categories)
class StorageKey(psa_storage.Key):
"""Representation of a key for storage format testing."""
IMPLICIT_USAGE_FLAGS = {
'PSA_KEY_USAGE_SIGN_HASH': 'PSA_KEY_USAGE_SIGN_MESSAGE',
'PSA_KEY_USAGE_VERIFY_HASH': 'PSA_KEY_USAGE_VERIFY_MESSAGE'
} #type: Dict[str, str]
"""Mapping of usage flags to the flags that they imply."""
def __init__(
self,
usage: Iterable[str],
without_implicit_usage: Optional[bool] = False,
**kwargs
) -> None:
"""Prepare to generate a key.
* `usage` : The usage flags used for the key.
* `without_implicit_usage`: Flag to define to apply the usage extension
"""
usage_flags = set(usage)
if not without_implicit_usage:
for flag in sorted(usage_flags):
if flag in self.IMPLICIT_USAGE_FLAGS:
usage_flags.add(self.IMPLICIT_USAGE_FLAGS[flag])
if usage_flags:
usage_expression = ' | '.join(sorted(usage_flags))
else:
usage_expression = '0'
super().__init__(usage=usage_expression, **kwargs)
class StorageTestData(StorageKey):
"""Representation of test case data for storage format testing."""
def __init__(
self,
description: str,
expected_usage: Optional[List[str]] = None,
**kwargs
) -> None:
"""Prepare to generate test data
* `description` : used for the test case names
* `expected_usage`: the usage flags generated as the expected usage flags
in the test cases. CAn differ from the usage flags
stored in the keys because of the usage flags extension.
"""
super().__init__(**kwargs)
self.description = description #type: str
if expected_usage is None:
self.expected_usage = self.usage #type: psa_storage.Expr
elif expected_usage:
self.expected_usage = psa_storage.Expr(' | '.join(expected_usage))
else:
self.expected_usage = psa_storage.Expr(0)
class StorageFormat:
"""Storage format stability test cases."""
def __init__(self, info: psa_information.Information, version: int, forward: bool) -> None:
"""Prepare to generate test cases for storage format stability.
* `info`: information about the API. See the `Information` class.
* `version`: the storage format version to generate test cases for.
* `forward`: if true, generate forward compatibility test cases which
save a key and check that its representation is as intended. Otherwise
generate backward compatibility test cases which inject a key
representation and check that it can be read and used.
"""
self.constructors = info.constructors #type: macro_collector.PSAMacroEnumerator
self.version = version #type: int
self.forward = forward #type: bool
RSA_OAEP_RE = re.compile(r'PSA_ALG_RSA_OAEP\((.*)\)\Z')
BRAINPOOL_RE = re.compile(r'PSA_KEY_TYPE_\w+\(PSA_ECC_FAMILY_BRAINPOOL_\w+\)\Z')
@classmethod
def exercise_key_with_algorithm(
cls,
key_type: psa_storage.Expr, bits: int,
alg: psa_storage.Expr
) -> bool:
"""Whether to exercise the given key with the given algorithm.
Normally only the type and algorithm matter for compatibility, and
this is handled in crypto_knowledge.KeyType.can_do(). This function
exists to detect exceptional cases. Exceptional cases detected here
are not tested in OpFail and should therefore have manually written
test cases.
"""
# Some test keys have the RAW_DATA type and attributes that don't
# necessarily make sense. We do this to validate numerical
# encodings of the attributes.
# Raw data keys have no useful exercise anyway so there is no
# loss of test coverage.
if key_type.string == 'PSA_KEY_TYPE_RAW_DATA':
return False
# OAEP requires room for two hashes plus wrapping
m = cls.RSA_OAEP_RE.match(alg.string)
if m:
hash_alg = m.group(1)
hash_length = crypto_knowledge.Algorithm.hash_length(hash_alg)
key_length = (bits + 7) // 8
# Leave enough room for at least one byte of plaintext
return key_length > 2 * hash_length + 2
# There's nothing wrong with ECC keys on Brainpool curves,
# but operations with them are very slow. So we only exercise them
# with a single algorithm, not with all possible hashes. We do
# exercise other curves with all algorithms so test coverage is
# perfectly adequate like this.
m = cls.BRAINPOOL_RE.match(key_type.string)
if m and alg.string != 'PSA_ALG_ECDSA_ANY':
return False
return True
def make_test_case(self, key: StorageTestData) -> test_case.TestCase:
"""Construct a storage format test case for the given key.
If ``forward`` is true, generate a forward compatibility test case:
create a key and validate that it has the expected representation.
Otherwise generate a backward compatibility test case: inject the
key representation into storage and validate that it can be read
correctly.
"""
verb = 'save' if self.forward else 'read'
tc = psa_test_case.TestCase()
tc.set_description(verb + ' ' + key.description)
tc.add_dependencies(psa_information.generate_deps_from_description(key.description))
tc.set_function('key_storage_' + verb)
tc.set_key_bits(key.bits)
tc.set_key_pair_usage(['IMPORT'] if self.forward else ['EXPORT'])
if self.forward:
extra_arguments = []
else:
flags = []
if self.exercise_key_with_algorithm(key.type, key.bits, key.alg):
flags.append('TEST_FLAG_EXERCISE')
if 'READ_ONLY' in key.lifetime.string:
flags.append('TEST_FLAG_READ_ONLY')
extra_arguments = [' | '.join(flags) if flags else '0']
tc.set_arguments([key.lifetime.string,
key.type.string, str(key.bits),
key.expected_usage.string,
key.alg.string, key.alg2.string,
'"' + key.material.hex() + '"',
'"' + key.hex() + '"',
*extra_arguments])
return tc
def key_for_lifetime(
self,
lifetime: str,
) -> StorageTestData:
"""Construct a test key for the given lifetime."""
short = lifetime
short = re.sub(r'PSA_KEY_LIFETIME_FROM_PERSISTENCE_AND_LOCATION',
r'', short)
short = crypto_knowledge.short_expression(short)
description = 'lifetime: ' + short
key = StorageTestData(version=self.version,
id=1, lifetime=lifetime,
type='PSA_KEY_TYPE_RAW_DATA', bits=8,
usage=['PSA_KEY_USAGE_EXPORT'], alg=0, alg2=0,
material=b'L',
description=description)
return key
def all_keys_for_lifetimes(self) -> Iterator[StorageTestData]:
"""Generate test keys covering lifetimes."""
lifetimes = sorted(self.constructors.lifetimes)
expressions = self.constructors.generate_expressions(lifetimes)
for lifetime in expressions:
# Don't attempt to create or load a volatile key in storage
if 'VOLATILE' in lifetime:
continue
# Don't attempt to create a read-only key in storage,
# but do attempt to load one.
if 'READ_ONLY' in lifetime and self.forward:
continue
yield self.key_for_lifetime(lifetime)
def key_for_usage_flags(
self,
usage_flags: List[str],
short: Optional[str] = None,
test_implicit_usage: Optional[bool] = True
) -> StorageTestData:
"""Construct a test key for the given key usage."""
extra_desc = ' without implication' if test_implicit_usage else ''
description = 'usage' + extra_desc + ': '
key1 = StorageTestData(version=self.version,
id=1, lifetime=0x00000001,
type='PSA_KEY_TYPE_RAW_DATA', bits=8,
expected_usage=usage_flags,
without_implicit_usage=not test_implicit_usage,
usage=usage_flags, alg=0, alg2=0,
material=b'K',
description=description)
if short is None:
usage_expr = key1.expected_usage.string
key1.description += crypto_knowledge.short_expression(usage_expr)
else:
key1.description += short
return key1
USAGE_FLAGS_NOT_VALID_IN_POLICIES = frozenset([
# Only for psa_check_key_usage() (upcoming) and
# mbedtls_pk_can_do_psa() (since TF-PSA-Crypto 1.0),
# not allowed in key policies as of TF-PSA-Crypto 1.0.
# Note that this may become dependent on the TF-PSA-Crypto version
# in the future; if so this code will require some refactoring.
'PSA_KEY_USAGE_DERIVE_PUBLIC',
])
def all_policy_flags(self) -> List[str]:
"""Return the list of all usage flags that are valid in key policies."""
known_flags = frozenset(self.constructors.key_usage_flags)
policy_flags = known_flags - self.USAGE_FLAGS_NOT_VALID_IN_POLICIES
return sorted(policy_flags)
def generate_keys_for_usage_flags(self, **kwargs) -> Iterator[StorageTestData]:
"""Generate test keys covering usage flags."""
policy_flags = self.all_policy_flags()
yield self.key_for_usage_flags(['0'], **kwargs)
for usage_flag in policy_flags:
yield self.key_for_usage_flags([usage_flag], **kwargs)
for flag1, flag2 in zip(policy_flags,
policy_flags[1:] + [policy_flags[0]]):
yield self.key_for_usage_flags([flag1, flag2], **kwargs)
def generate_key_for_all_usage_flags(self) -> Iterator[StorageTestData]:
policy_flags = self.all_policy_flags()
yield self.key_for_usage_flags(policy_flags, short='all valid')
def all_keys_for_usage_flags(self) -> Iterator[StorageTestData]:
yield from self.generate_keys_for_usage_flags()
yield from self.generate_key_for_all_usage_flags()
def key_for_type_and_alg(
self,
kt: crypto_knowledge.KeyType,
bits: int,
alg: Optional[crypto_knowledge.Algorithm] = None,
) -> StorageTestData:
"""Construct a test key of the given type.
If alg is not None, this key allows it.
"""
usage_flags = ['PSA_KEY_USAGE_EXPORT']
alg1 = 0 #type: psa_storage.Exprable
alg2 = 0
if alg is not None:
alg1 = alg.expression
usage_flags += alg.usage_flags(public=kt.is_public())
key_material = kt.key_material(bits)
description = 'type: {} {}-bit'.format(kt.short_expression(1), bits)
if alg is not None:
description += ', ' + alg.short_expression(1)
key = StorageTestData(version=self.version,
id=1, lifetime=0x00000001,
type=kt.expression, bits=bits,
usage=usage_flags, alg=alg1, alg2=alg2,
material=key_material,
description=description)
return key
def keys_for_type(
self,
key_type: str,
all_algorithms: List[crypto_knowledge.Algorithm],
) -> Iterator[StorageTestData]:
"""Generate test keys for the given key type."""
kt = crypto_knowledge.KeyType(key_type)
for bits in kt.sizes_to_test():
# Test a non-exercisable key, as well as exercisable keys for
# each compatible algorithm.
# To do: test reading a key from storage with an incompatible
# or unsupported algorithm.
yield self.key_for_type_and_alg(kt, bits)
compatible_algorithms = [alg for alg in all_algorithms
if kt.can_do(alg)]
for alg in compatible_algorithms:
if alg.expression == 'PSA_ALG_XTS':
# XTS mode uses double-size keys for the underlying block cipher
# XTS does not use 192-bit keys
if bits != 192:
bits = bits * 2
else:
continue
yield self.key_for_type_and_alg(kt, bits, alg)
def all_keys_for_types(self) -> Iterator[StorageTestData]:
"""Generate test keys covering key types and their representations."""
key_types = sorted(self.constructors.key_types)
all_algorithms = [crypto_knowledge.Algorithm(alg)
for alg in self.constructors.generate_expressions(
sorted(self.constructors.algorithms)
)]
for key_type in self.constructors.generate_expressions(key_types):
yield from self.keys_for_type(key_type, all_algorithms)
def keys_for_algorithm(self, alg: str) -> Iterator[StorageTestData]:
"""Generate test keys for the encoding of the specified algorithm."""
# These test cases only validate the encoding of algorithms, not
# whether the key read from storage is suitable for an operation.
# `keys_for_types` generate read tests with an algorithm and a
# compatible key.
descr = crypto_knowledge.short_expression(alg, 1)
usage = ['PSA_KEY_USAGE_EXPORT']
key1 = StorageTestData(version=self.version,
id=1, lifetime=0x00000001,
type='PSA_KEY_TYPE_RAW_DATA', bits=8,
usage=usage, alg=alg, alg2=0,
material=b'K',
description='alg: ' + descr)
yield key1
key2 = StorageTestData(version=self.version,
id=1, lifetime=0x00000001,
type='PSA_KEY_TYPE_RAW_DATA', bits=8,
usage=usage, alg=0, alg2=alg,
material=b'L',
description='alg2: ' + descr)
yield key2
def all_keys_for_algorithms(self) -> Iterator[StorageTestData]:
"""Generate test keys covering algorithm encodings."""
algorithms = sorted(self.constructors.algorithms)
for alg in self.constructors.generate_expressions(algorithms):
yield from self.keys_for_algorithm(alg)
def generate_all_keys(self) -> Iterator[StorageTestData]:
"""Generate all keys for the test cases."""
yield from self.all_keys_for_lifetimes()
yield from self.all_keys_for_usage_flags()
yield from self.all_keys_for_types()
yield from self.all_keys_for_algorithms()
def all_test_cases(self) -> Iterator[test_case.TestCase]:
"""Generate all storage format test cases."""
# First build a list of all keys, then construct all the corresponding
# test cases. This allows all required information to be obtained in
# one go, which is a significant performance gain as the information
# includes numerical values obtained by compiling a C program.
all_keys = list(self.generate_all_keys())
for key in all_keys:
if key.location_value() != 0:
# Skip keys with a non-default location, because they
# require a driver and we currently have no mechanism to
# determine whether a driver is available.
continue
yield self.make_test_case(key)
class StorageFormatForward(StorageFormat):
"""Storage format stability test cases for forward compatibility."""
def __init__(self, info: psa_information.Information, version: int) -> None:
super().__init__(info, version, True)
class StorageFormatV0(StorageFormat):
"""Storage format stability test cases for version 0 compatibility."""
def __init__(self, info: psa_information.Information) -> None:
super().__init__(info, 0, False)
def all_keys_for_usage_flags(self) -> Iterator[StorageTestData]:
"""Generate test keys covering usage flags."""
yield from super().all_keys_for_usage_flags()
yield from self.generate_keys_for_usage_flags(test_implicit_usage=False)
def keys_for_implicit_usage(
self,
implyer_usage: str,
alg: str,
key_type: crypto_knowledge.KeyType
) -> StorageTestData:
# pylint: disable=too-many-locals
"""Generate test keys for the specified implicit usage flag,
algorithm and key type combination.
"""
bits = key_type.sizes_to_test()[0]
implicit_usage = StorageKey.IMPLICIT_USAGE_FLAGS[implyer_usage]
usage_flags = ['PSA_KEY_USAGE_EXPORT']
material_usage_flags = usage_flags + [implyer_usage]
expected_usage_flags = material_usage_flags + [implicit_usage]
alg2 = 0
key_material = key_type.key_material(bits)
usage_expression = crypto_knowledge.short_expression(implyer_usage, 1)
alg_expression = crypto_knowledge.short_expression(alg, 1)
key_type_expression = key_type.short_expression(1)
description = 'implied by {}: {} {} {}-bit'.format(
usage_expression, alg_expression, key_type_expression, bits)
key = StorageTestData(version=self.version,
id=1, lifetime=0x00000001,
type=key_type.expression, bits=bits,
usage=material_usage_flags,
expected_usage=expected_usage_flags,
without_implicit_usage=True,
alg=alg, alg2=alg2,
material=key_material,
description=description)
return key
def gather_key_types_for_sign_alg(self) -> Dict[str, List[str]]:
# pylint: disable=too-many-locals
"""Match possible key types for sign algorithms."""
# To create a valid combination both the algorithms and key types
# must be filtered. Pair them with keywords created from its names.
incompatible_alg_keyword = frozenset(['RAW', 'ANY', 'PURE'])
incompatible_key_type_keywords = frozenset(['MONTGOMERY'])
keyword_translation = {
'ECDSA': 'ECC',
'ED[0-9]*.*' : 'EDWARDS'
}
exclusive_keywords = {
'EDWARDS': 'ECC'
}
key_types = set(self.constructors.generate_expressions(self.constructors.key_types))
algorithms = set(self.constructors.generate_expressions(self.constructors.sign_algorithms))
alg_with_keys = {} #type: Dict[str, List[str]]
translation_table = str.maketrans('(', '_', ')')
for alg in algorithms:
# Generate keywords from the name of the algorithm
alg_keywords = set(alg.partition('(')[0].split(sep='_')[2:])
# Translate keywords for better matching with the key types
for keyword in alg_keywords.copy():
for pattern, replace in keyword_translation.items():
if re.match(pattern, keyword):
alg_keywords.remove(keyword)
alg_keywords.add(replace)
# Filter out incompatible algorithms
if not alg_keywords.isdisjoint(incompatible_alg_keyword):
continue
for key_type in key_types:
# Generate keywords from the of the key type
key_type_keywords = set(key_type.translate(translation_table).split(sep='_')[3:])
# Remove ambiguous keywords
for keyword1, keyword2 in exclusive_keywords.items():
if keyword1 in key_type_keywords:
key_type_keywords.remove(keyword2)
if key_type_keywords.isdisjoint(incompatible_key_type_keywords) and\
not key_type_keywords.isdisjoint(alg_keywords):
if alg in alg_with_keys:
alg_with_keys[alg].append(key_type)
else:
alg_with_keys[alg] = [key_type]
return alg_with_keys
def all_keys_for_implicit_usage(self) -> Iterator[StorageTestData]:
"""Generate test keys for usage flag extensions."""
# Generate a key type and algorithm pair for each extendable usage
# flag to generate a valid key for exercising. The key is generated
# without usage extension to check the extension compatibility.
alg_with_keys = self.gather_key_types_for_sign_alg()
for usage in sorted(StorageKey.IMPLICIT_USAGE_FLAGS, key=str):
for alg in sorted(alg_with_keys):
for key_type in sorted(alg_with_keys[alg]):
# The key types must be filtered to fit the specific usage flag.
kt = crypto_knowledge.KeyType(key_type)
if kt.is_public() and '_SIGN_' in usage:
# Can't sign with a public key
continue
yield self.keys_for_implicit_usage(usage, alg, kt)
def generate_all_keys(self) -> Iterator[StorageTestData]:
yield from super().generate_all_keys()
yield from self.all_keys_for_implicit_usage()
class PSATestGenerator(test_data_generation.TestGenerator):
"""Test generator subclass including PSA targets and info."""
# Note that targets whose names contain 'test_format' have their content
# validated by `abi_check.py`.
targets = {
'test_suite_psa_crypto_generate_key.generated':
lambda info: KeyGenerate(info).test_cases_for_key_generation(),
'test_suite_psa_crypto_not_supported.generated':
lambda info: KeyTypeNotSupported(info).test_cases_for_not_supported(),
'test_suite_psa_crypto_low_hash.generated':
lambda info: crypto_data_tests.HashPSALowLevel(info).all_test_cases(),
'test_suite_psa_crypto_op_fail.generated':
lambda info: OpFail(info).all_test_cases(),
'test_suite_psa_crypto_storage_format.current':
lambda info: StorageFormatForward(info, 0).all_test_cases(),
'test_suite_psa_crypto_storage_format.v0':
lambda info: StorageFormatV0(info).all_test_cases(),
} #type: Dict[str, Callable[[psa_information.Information], Iterable[test_case.TestCase]]]
def __init__(self, options):
super().__init__(options)
self.info = psa_information.Information()
def generate_target(self, name: str, *target_args) -> None:
super().generate_target(name, self.info)
if __name__ == '__main__':
test_data_generation.main(sys.argv[1:], __doc__, PSATestGenerator)

View File

@@ -0,0 +1,51 @@
#!/usr/bin/env python3
"""Generate wrapper functions for PSA function calls.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
from mbedtls_framework.code_wrapper.psa_test_wrapper import PSATestWrapper, PSALoggingTestWrapper
from mbedtls_framework import build_tree
def main() -> None:
default_c_output_file_name = 'tests/src/psa_test_wrappers.c'
default_h_output_file_name = 'tests/include/test/psa_test_wrappers.h'
project_root = build_tree.guess_project_root()
if build_tree.looks_like_mbedtls_root(project_root) and \
not build_tree.is_mbedtls_3_6():
default_c_output_file_name = 'tf-psa-crypto/' + default_c_output_file_name
default_h_output_file_name = 'tf-psa-crypto/' + default_h_output_file_name
parser = argparse.ArgumentParser(description=globals()['__doc__'])
parser.add_argument('--log',
help='Stream to log to (default: no logging code)')
parser.add_argument('--output-c',
metavar='FILENAME',
default=default_c_output_file_name,
help=('Output .c file path (default: {}; skip .c output if empty)'
.format(default_c_output_file_name)))
parser.add_argument('--output-h',
metavar='FILENAME',
default=default_h_output_file_name,
help=('Output .h file path (default: {}; skip .h output if empty)'
.format(default_h_output_file_name)))
options = parser.parse_args()
if options.log:
generator = PSALoggingTestWrapper(default_h_output_file_name,
default_c_output_file_name,
options.log) #type: PSATestWrapper
else:
generator = PSATestWrapper(default_h_output_file_name,
default_c_output_file_name)
if options.output_h:
generator.write_h_file(options.output_h)
if options.output_c:
generator.write_c_file(options.output_c)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,87 @@
#!/usr/bin/env python3
"""Generate server9-bad-saltlen.crt
Generate a certificate signed with RSA-PSS, with an incorrect salt length.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import subprocess
import argparse
from asn1crypto import pem, x509, core #type: ignore #pylint: disable=import-error
OPENSSL_RSA_PSS_CERT_COMMAND = r'''
openssl x509 -req -CA {ca_name}.crt -CAkey {ca_name}.key -set_serial 24 {ca_password} \
{openssl_extfile} -days 3650 -outform DER -in {csr} \
-sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:{anounce_saltlen} \
-sigopt rsa_mgf1_md:sha256
'''
SIG_OPT = \
r'-sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:{saltlen} -sigopt rsa_mgf1_md:sha256'
OPENSSL_RSA_PSS_DGST_COMMAND = r'''openssl dgst -sign {ca_name}.key {ca_password} \
-sigopt rsa_padding_mode:pss -sigopt rsa_pss_saltlen:{actual_saltlen} \
-sigopt rsa_mgf1_md:sha256'''
def auto_int(x):
return int(x, 0)
def build_argparser(parser):
"""Build argument parser"""
parser.description = __doc__
parser.add_argument('--ca-name', type=str, required=True,
help='Basename of CA files')
parser.add_argument('--ca-password', type=str,
required=True, help='CA key file password')
parser.add_argument('--csr', type=str, required=True,
help='CSR file for generating certificate')
parser.add_argument('--openssl-extfile', type=str,
required=True, help='X905 v3 extension config file')
parser.add_argument('--anounce_saltlen', type=auto_int,
required=True, help='Announced salt length')
parser.add_argument('--actual_saltlen', type=auto_int,
required=True, help='Actual salt length')
parser.add_argument('--output', type=str, required=True)
def main():
parser = argparse.ArgumentParser()
build_argparser(parser)
args = parser.parse_args()
return generate(**vars(args))
def generate(**kwargs):
"""Generate different salt length certificate file."""
ca_password = kwargs.get('ca_password', '')
if ca_password:
kwargs['ca_password'] = r'-passin "pass:{ca_password}"'.format(
**kwargs)
else:
kwargs['ca_password'] = ''
extfile = kwargs.get('openssl_extfile', '')
if extfile:
kwargs['openssl_extfile'] = '-extfile {openssl_extfile}'.format(
**kwargs)
else:
kwargs['openssl_extfile'] = ''
cmd = OPENSSL_RSA_PSS_CERT_COMMAND.format(**kwargs)
der_bytes = subprocess.check_output(cmd, shell=True)
target_certificate = x509.Certificate.load(der_bytes)
cmd = OPENSSL_RSA_PSS_DGST_COMMAND.format(**kwargs)
#pylint: disable=unexpected-keyword-arg
der_bytes = subprocess.check_output(cmd,
input=target_certificate['tbs_certificate'].dump(),
shell=True)
with open(kwargs.get('output'), 'wb') as f:
target_certificate['signature_value'] = core.OctetBitString(der_bytes)
f.write(pem.armor('CERTIFICATE', target_certificate.dump()))
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,415 @@
#!/usr/bin/env python3
"""Generate library/ssl_debug_helpers_generated.c
The code generated by this module includes debug helper functions that can not be
implemented by fixed codes.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import sys
import re
import os
import textwrap
import argparse
from mbedtls_framework import build_tree
def remove_c_comments(string):
"""
Remove C style comments from input string
"""
string_pattern = r"(?P<string>\".*?\"|\'.*?\')"
comment_pattern = r"(?P<comment>/\*.*?\*/|//[^\r\n]*$)"
pattern = re.compile(string_pattern + r'|' + comment_pattern,
re.MULTILINE | re.DOTALL)
def replacer(match):
if match.lastgroup == 'comment':
return ""
return match.group()
return pattern.sub(replacer, string)
class CondDirectiveNotMatch(Exception):
pass
def preprocess_c_source_code(source, *classes):
"""
Simple preprocessor for C source code.
Only processes condition directives without expanding them.
Yield object according to the classes input. Most match firstly
If the directive pair does not match , raise CondDirectiveNotMatch.
Assume source code does not include comments and compile pass.
"""
pattern = re.compile(r"^[ \t]*#[ \t]*" +
r"(?P<directive>(if[ \t]|ifndef[ \t]|ifdef[ \t]|else|endif))" +
r"[ \t]*(?P<param>(.*\\\n)*.*$)",
re.MULTILINE)
stack = []
def _yield_objects(s, d, p, st, end):
"""
Output matched source piece
"""
nonlocal stack
start_line, end_line = '', ''
if stack:
start_line = '#{} {}'.format(d, p)
if d == 'if':
end_line = '#endif /* {} */'.format(p)
elif d == 'ifdef':
end_line = '#endif /* defined({}) */'.format(p)
else:
end_line = '#endif /* !defined({}) */'.format(p)
has_instance = False
for cls in classes:
for instance in cls.extract(s, st, end):
if has_instance is False:
has_instance = True
yield pair_start, start_line
yield instance.span()[0], instance
if has_instance:
yield start, end_line
for match in pattern.finditer(source):
directive = match.groupdict()['directive'].strip()
param = match.groupdict()['param']
start, end = match.span()
if directive in ('if', 'ifndef', 'ifdef'):
stack.append((directive, param, start, end))
continue
if not stack:
raise CondDirectiveNotMatch()
pair_directive, pair_param, pair_start, pair_end = stack.pop()
yield from _yield_objects(source,
pair_directive,
pair_param,
pair_end,
start)
if directive == 'endif':
continue
if pair_directive == 'if':
directive = 'if'
param = "!( {} )".format(pair_param)
elif pair_directive == 'ifdef':
directive = 'ifndef'
param = pair_param
else:
directive = 'ifdef'
param = pair_param
stack.append((directive, param, start, end))
assert not stack, len(stack)
class EnumDefinition:
"""
Generate helper functions around enumeration.
Currently, it generate translation function from enum value to string.
Enum definition looks like:
[typedef] enum [prefix name] { [body] } [suffix name];
Known limitation:
- the '}' and ';' SHOULD NOT exist in different macro blocks. Like
```
enum test {
....
#if defined(A)
....
};
#else
....
};
#endif
```
"""
@classmethod
def extract(cls, source_code, start=0, end=-1):
enum_pattern = re.compile(r'enum\s*(?P<prefix_name>\w*)\s*' +
r'{\s*(?P<body>[^}]*)}' +
r'\s*(?P<suffix_name>\w*)\s*;',
re.MULTILINE | re.DOTALL)
for match in enum_pattern.finditer(source_code, start, end):
yield EnumDefinition(source_code,
span=match.span(),
group=match.groupdict())
def __init__(self, source_code, span=None, group=None):
assert isinstance(group, dict)
prefix_name = group.get('prefix_name', None)
suffix_name = group.get('suffix_name', None)
body = group.get('body', None)
assert prefix_name or suffix_name
assert body
assert span
# If suffix_name exists, it is a typedef
self._prototype = suffix_name if suffix_name else 'enum ' + prefix_name
self._name = suffix_name if suffix_name else prefix_name
self._body = body
self._source = source_code
self._span = span
def __repr__(self):
return 'Enum({},{})'.format(self._name, self._span)
def __str__(self):
return repr(self)
def span(self):
return self._span
def generate_translation_function(self):
"""
Generate function for translating value to string
"""
translation_table = []
for line in self._body.splitlines():
if line.strip().startswith('#'):
# Preprocess directive, keep it in table
translation_table.append(line.strip())
continue
if not line.strip():
continue
for field in line.strip().split(','):
if not field.strip():
continue
member = field.strip().split()[0]
translation_table.append(
'{space}case {member}:\n{space} return "{member}";'
.format(member=member, space=' '*8)
)
body = textwrap.dedent('''\
const char *{name}_str( {prototype} in )
{{
switch (in) {{
{translation_table}
default:
return "UNKNOWN_VALUE";
}}
}}
''')
body = body.format(translation_table='\n'.join(translation_table),
name=self._name,
prototype=self._prototype)
return body
class SignatureAlgorithmDefinition:
"""
Generate helper functions for signature algorithms.
It generates translation function from signature algorithm define to string.
Signature algorithm definition looks like:
#define MBEDTLS_TLS1_3_SIG_[ upper case signature algorithm ] [ value(hex) ]
Known limitation:
- the definitions SHOULD exist in same macro blocks.
"""
@classmethod
def extract(cls, source_code, start=0, end=-1):
sig_alg_pattern = re.compile(r'#define\s+(?P<name>MBEDTLS_TLS1_3_SIG_\w+)\s+' +
r'(?P<value>0[xX][0-9a-fA-F]+)$',
re.MULTILINE | re.DOTALL)
matches = list(sig_alg_pattern.finditer(source_code, start, end))
if matches:
yield SignatureAlgorithmDefinition(source_code, definitions=matches)
def __init__(self, source_code, definitions=None):
if definitions is None:
definitions = []
assert isinstance(definitions, list) and definitions
self._definitions = definitions
self._source = source_code
def __repr__(self):
return 'SigAlgs({})'.format(self._definitions[0].span())
def span(self):
return self._definitions[0].span()
def __str__(self):
"""
Generate function for translating value to string
"""
translation_table = []
for m in self._definitions:
name = m.groupdict()['name']
return_val = name[len('MBEDTLS_TLS1_3_SIG_'):].lower()
translation_table.append(
' case {}:\n return "{}";'.format(name, return_val))
body = textwrap.dedent('''\
const char *mbedtls_ssl_sig_alg_to_str( uint16_t in )
{{
switch( in )
{{
{translation_table}
}};
return "UNKNOWN";
}}''')
body = body.format(translation_table='\n'.join(translation_table))
return body
class NamedGroupDefinition:
"""
Generate helper functions for named group
It generates translation function from named group define to string.
Named group definition looks like:
#define MBEDTLS_SSL_IANA_TLS_GROUP_[ upper case named group ] [ value(hex) ]
Known limitation:
- the definitions SHOULD exist in same macro blocks.
"""
@classmethod
def extract(cls, source_code, start=0, end=-1):
named_group_pattern = re.compile(r'#define\s+(?P<name>MBEDTLS_SSL_IANA_TLS_GROUP_\w+)\s+' +
r'(?P<value>0[xX][0-9a-fA-F]+)$',
re.MULTILINE | re.DOTALL)
matches = list(named_group_pattern.finditer(source_code, start, end))
if matches:
yield NamedGroupDefinition(source_code, definitions=matches)
def __init__(self, source_code, definitions=None):
if definitions is None:
definitions = []
assert isinstance(definitions, list) and definitions
self._definitions = definitions
self._source = source_code
def __repr__(self):
return 'NamedGroup({})'.format(self._definitions[0].span())
def span(self):
return self._definitions[0].span()
def __str__(self):
"""
Generate function for translating value to string
"""
translation_table = []
for m in self._definitions:
name = m.groupdict()['name']
iana_name = name[len('MBEDTLS_SSL_IANA_TLS_GROUP_'):].lower()
translation_table.append(' case {}:\n return "{}";'.format(name, iana_name))
body = textwrap.dedent('''\
const char *mbedtls_ssl_named_group_to_str( uint16_t in )
{{
switch( in )
{{
{translation_table}
}};
return "UNKNOWN";
}}''')
body = body.format(translation_table='\n'.join(translation_table))
return body
OUTPUT_C_TEMPLATE = '''\
/* Automatically generated by generate_ssl_debug_helpers.py. DO NOT EDIT. */
/**
* \\file ssl_debug_helpers_generated.c
*
* \\brief Automatically generated helper functions for debugging
*/
/*
* Copyright The Mbed TLS Contributors
* SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
*
*/
#include "ssl_misc.h"
#if defined(MBEDTLS_DEBUG_C)
#include "ssl_debug_helpers.h"
{functions}
#endif /* MBEDTLS_DEBUG_C */
/* End of automatically generated file. */
'''
def generate_ssl_debug_helpers(output_directory, mbedtls_root):
"""
Generate functions of debug helps
"""
mbedtls_root = os.path.abspath(
mbedtls_root or build_tree.guess_mbedtls_root())
with open(os.path.join(mbedtls_root, 'include/mbedtls/ssl.h')) as f:
source_code = remove_c_comments(f.read())
definitions = dict()
for start, instance in preprocess_c_source_code(source_code,
EnumDefinition,
SignatureAlgorithmDefinition,
NamedGroupDefinition):
if start in definitions:
continue
if isinstance(instance, EnumDefinition):
definition = instance.generate_translation_function()
else:
definition = instance
definitions[start] = definition
function_definitions = [str(v) for _, v in sorted(definitions.items())]
if output_directory == sys.stdout:
sys.stdout.write(OUTPUT_C_TEMPLATE.format(
functions='\n'.join(function_definitions)))
else:
with open(os.path.join(output_directory, 'ssl_debug_helpers_generated.c'), 'w') as f:
f.write(OUTPUT_C_TEMPLATE.format(
functions='\n'.join(function_definitions)))
def main():
"""
Command line entry
"""
parser = argparse.ArgumentParser()
parser.add_argument('--mbedtls-root', nargs='?', default=None,
help='root directory of mbedtls source code')
parser.add_argument('output_directory', nargs='?',
default='library', help='source/header files location')
args = parser.parse_args()
generate_ssl_debug_helpers(args.output_directory, args.mbedtls_root)
return 0
if __name__ == '__main__':
sys.exit(main())

View File

@@ -0,0 +1,109 @@
#!/usr/bin/env python3
"""
Generate `tests/src/test_certs.h` which includes certficaties/keys/certificate list for testing.
"""
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import os
import sys
import argparse
import jinja2
from mbedtls_framework.build_tree import guess_project_root
TESTS_DIR = os.path.join(guess_project_root(), 'tests')
FRAMEWORK_DIR = os.path.join(guess_project_root(), 'framework')
DATA_FILES_PATH = os.path.join(FRAMEWORK_DIR, 'data_files')
INPUT_ARGS = [
("string", "TEST_CA_CRT_EC_PEM", DATA_FILES_PATH + "/test-ca2.crt"),
("binary", "TEST_CA_CRT_EC_DER", DATA_FILES_PATH + "/test-ca2.crt.der"),
("string", "TEST_CA_KEY_EC_PEM", DATA_FILES_PATH + "/test-ca2.key.enc"),
("password", "TEST_CA_PWD_EC_PEM", "PolarSSLTest"),
("binary", "TEST_CA_KEY_EC_DER", DATA_FILES_PATH + "/test-ca2.key.der"),
("string", "TEST_CA_CRT_RSA_SHA256_PEM", DATA_FILES_PATH + "/test-ca-sha256.crt"),
("binary", "TEST_CA_CRT_RSA_SHA256_DER", DATA_FILES_PATH + "/test-ca-sha256.crt.der"),
("string", "TEST_CA_CRT_RSA_SHA1_PEM", DATA_FILES_PATH + "/test-ca-sha1.crt"),
("binary", "TEST_CA_CRT_RSA_SHA1_DER", DATA_FILES_PATH + "/test-ca-sha1.crt.der"),
("string", "TEST_CA_KEY_RSA_PEM", DATA_FILES_PATH + "/test-ca.key"),
("password", "TEST_CA_PWD_RSA_PEM", "PolarSSLTest"),
("binary", "TEST_CA_KEY_RSA_DER", DATA_FILES_PATH + "/test-ca.key.der"),
("string", "TEST_SRV_CRT_EC_PEM", DATA_FILES_PATH + "/server5.crt"),
("binary", "TEST_SRV_CRT_EC_DER", DATA_FILES_PATH + "/server5.crt.der"),
("string", "TEST_SRV_KEY_EC_PEM", DATA_FILES_PATH + "/server5.key"),
("binary", "TEST_SRV_KEY_EC_DER", DATA_FILES_PATH + "/server5.key.der"),
("string", "TEST_SRV_CRT_RSA_SHA256_PEM", DATA_FILES_PATH + "/server2-sha256.crt"),
("binary", "TEST_SRV_CRT_RSA_SHA256_DER", DATA_FILES_PATH + "/server2-sha256.crt.der"),
("string", "TEST_SRV_CRT_RSA_SHA1_PEM", DATA_FILES_PATH + "/server2.crt"),
("binary", "TEST_SRV_CRT_RSA_SHA1_DER", DATA_FILES_PATH + "/server2.crt.der"),
("string", "TEST_SRV_KEY_RSA_PEM", DATA_FILES_PATH + "/server2.key"),
("binary", "TEST_SRV_KEY_RSA_DER", DATA_FILES_PATH + "/server2.key.der"),
("string", "TEST_CLI_CRT_EC_PEM", DATA_FILES_PATH + "/cli2.crt"),
("binary", "TEST_CLI_CRT_EC_DER", DATA_FILES_PATH + "/cli2.crt.der"),
("string", "TEST_CLI_KEY_EC_PEM", DATA_FILES_PATH + "/cli2.key"),
("binary", "TEST_CLI_KEY_EC_DER", DATA_FILES_PATH + "/cli2.key.der"),
("string", "TEST_CLI_CRT_RSA_PEM", DATA_FILES_PATH + "/cli-rsa-sha256.crt"),
("binary", "TEST_CLI_CRT_RSA_DER", DATA_FILES_PATH + "/cli-rsa-sha256.crt.der"),
("string", "TEST_CLI_KEY_RSA_PEM", DATA_FILES_PATH + "/cli-rsa.key"),
("binary", "TEST_CLI_KEY_RSA_DER", DATA_FILES_PATH + "/cli-rsa.key.der"),
]
def main():
parser = argparse.ArgumentParser()
default_output_path = os.path.join(TESTS_DIR, 'include', 'test', 'test_certs.h')
parser.add_argument('--output', type=str, default=default_output_path)
parser.add_argument('--list-dependencies', action='store_true')
args = parser.parse_args()
if args.list_dependencies:
files_list = [arg[2] for arg in INPUT_ARGS
if arg[0] != "password"]
print(" ".join(files_list))
return
generate(INPUT_ARGS, output=args.output)
#pylint: disable=dangerous-default-value, unused-argument
def generate(values=[], output=None):
"""Generate C header file.
"""
template_loader = jinja2.FileSystemLoader(DATA_FILES_PATH)
template_env = jinja2.Environment(
loader=template_loader, lstrip_blocks=True, trim_blocks=True,
keep_trailing_newline=True)
def read_as_c_array(filename):
with open(filename, 'rb') as f:
data = f.read(12)
while data:
yield ', '.join(['{:#04x}'.format(b) for b in data])
data = f.read(12)
def read_lines(filename):
with open(filename) as f:
try:
for line in f:
yield line.strip()
except:
print(filename)
raise
def put_to_column(value, position=0):
return ' '*position + value
template_env.filters['read_as_c_array'] = read_as_c_array
template_env.filters['read_lines'] = read_lines
template_env.filters['put_to_column'] = put_to_column
template = template_env.get_template('test_certs.h.jinja2')
with open(output, 'w') as f:
f.write(template.render(macros=values))
if __name__ == '__main__':
sys.exit(main())

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,221 @@
#!/usr/bin/env python3
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
"""Module generating EC and RSA keys to be used in test_suite_pk instead of
generating the required key at run time. This helps speeding up testing."""
from typing import Iterator, List, Tuple
import re
import argparse
from mbedtls_framework.asymmetric_key_data import ASYMMETRIC_KEY_DATA
from mbedtls_framework import build_tree
BYTES_PER_LINE = 16
def c_byte_array_literal_content(array_name: str, key_data: bytes) -> Iterator[str]:
"""Return C code that defines array_name as a byte array with the given content."""
yield 'static const unsigned char '
yield array_name
yield '[] = {'
for index in range(0, len(key_data), BYTES_PER_LINE):
yield '\n '
for b in key_data[index:index + BYTES_PER_LINE]:
yield ' {:#04x},'.format(b)
yield '\n};'
def convert_der_to_c(array_name: str, key_data: bytes) -> str:
return ''.join(c_byte_array_literal_content(array_name, key_data))
def get_key_type(key_type: str) -> str:
"""Short name for a PSA key type."""
if key_type.startswith('PSA_KEY_TYPE_ECC_'):
return "ec"
elif key_type.startswith('PSA_KEY_TYPE_ML_DSA_'):
return "mldsa"
elif key_type.startswith('PSA_KEY_TYPE_ML_KEM_'):
return "mlkem"
elif key_type.startswith('PSA_KEY_TYPE_RSA_'):
return "rsa"
elif key_type.startswith('PSA_KEY_TYPE_SLH_DSA_'):
return "slhdsa"
else:
raise Exception(f"Unhandled key type {key_type}")
def get_ec_key_family(key: str) -> str:
"""Extract "PSA_ECC_xxx" from "PSA_KEY_TYPE_ECC_ttt(PSA_ECC_xxx)"."""
match = re.search(r'.*\((.*)\)', key)
if match is None:
raise Exception("Unable to get EC family from {}".format(key))
return match.group(1)
# Legacy EC group ID do not support all the key types that PSA does, so the
# following dictionaries are used for:
# - getting prefix/suffix for legacy curve names
# - understand if the curve is supported in legacy symbols (MBEDTLS_ECP_DP_...)
EC_NAME_CONVERSION = {
'PSA_ECC_FAMILY_SECP_K1': {
192: ('secp', 'k1'),
256: ('secp', 'k1')
},
'PSA_ECC_FAMILY_SECP_R1': {
192: ('secp', 'r1'),
224: ('secp', 'r1'),
256: ('secp', 'r1'),
384: ('secp', 'r1'),
521: ('secp', 'r1')
},
'PSA_ECC_FAMILY_BRAINPOOL_P_R1': {
256: ('bp', 'r1'),
384: ('bp', 'r1'),
512: ('bp', 'r1')
},
'PSA_ECC_FAMILY_MONTGOMERY': {
255: ('curve', '19'),
448: ('curve', '')
}
}
def get_ec_curve_name(priv_key: str, bits: int) -> str:
"""Short name for an elliptic curve key type."""
ec_family = get_ec_key_family(priv_key)
try:
prefix = EC_NAME_CONVERSION[ec_family][bits][0]
suffix = EC_NAME_CONVERSION[ec_family][bits][1]
except KeyError:
return ""
return prefix + str(bits) + suffix
def get_slh_dsa_family(key_type: str) -> str:
"""Short name from an SLH-DSA family."""
m = re.search(r'PSA_SLH_FAMILY_(\w+)', key_type)
assert m
return m.group(1).replace('_', '').lower()
def get_look_up_table_entry(key_type: str, group_id_or_keybits: str,
priv_array_name: str, pub_array_name: str) -> Iterator[str]:
"""Yield C code lines for the definition of a key pair and its matching public key."""
if key_type == "ec":
yield " {{ {}, 0,\n".format(group_id_or_keybits)
else:
yield " {{ 0, {},\n".format(group_id_or_keybits)
yield " {0}, sizeof({0}),\n".format(priv_array_name)
yield " {0}, sizeof({0}) }},".format(pub_array_name)
def write_output_file(output_file_name: str, arrays: str, look_up_table: str):
"""Write generated content to the output file"""
with open(output_file_name, 'wt') as output:
output.write("""\
/*********************************************************************************
* This file was automatically generated from framework/scripts/generate_test_keys.py.
* Please do not edit it manually.
*********************************************************************************/
#ifndef TEST_TEST_KEYS_H
#define TEST_TEST_KEYS_H
#if TF_PSA_CRYPTO_VERSION_MAJOR >= 1
#include <tf_psa_crypto_common.h>
#include <mbedtls/private/ecp.h>
#else
#include <common.h>
#include <mbedtls/ecp.h>
#endif
""")
output.write(arrays)
output.write("""
struct predefined_key_element {{
int group_id; // EC group ID; 0 for RSA keys
int keybits; // bits size of RSA key; 0 for EC keys
const unsigned char *priv_key;
size_t priv_key_len;
const unsigned char *pub_key;
size_t pub_key_len;
}};
MBEDTLS_MAYBE_UNUSED static struct predefined_key_element predefined_keys[] = {{
{}
}};
#endif /* TEST_TEST_KEYS_H */
/* End of generated file */
""".format(look_up_table))
def collect_keys() -> Tuple[str, str]:
""""
This function reads key data from ASYMMETRIC_KEY_DATA and, only for the
keys supported in legacy ECP/RSA modules, it returns 2 strings:
- the 1st contains C arrays declaration of these keys and
- the 2nd contains the final look-up table for all these arrays.
"""
arrays = []
look_up_table = []
# Get a list of private keys only in order to get a single item for every
# (key type, key bits) pair. We know that ASYMMETRIC_KEY_DATA
# contains also the public counterpart.
priv_keys = [key for key in ASYMMETRIC_KEY_DATA if '_KEY_PAIR' in key]
priv_keys = sorted(priv_keys)
for priv_key in priv_keys:
key_type = get_key_type(priv_key)
pub_key = re.sub('_KEY_PAIR', '_PUBLIC_KEY', priv_key)
for bits in ASYMMETRIC_KEY_DATA[priv_key]:
if key_type == "ec":
curve = get_ec_curve_name(priv_key, bits)
# Ignore EC curves unsupported in legacy symbols
if curve == "":
continue
# Create output array name
if key_type == "ec":
array_name_base = "_".join(["test", key_type, curve])
elif key_type == "slhdsa":
family = get_slh_dsa_family(priv_key)
array_name_base = "_".join(["test", key_type, family, str(bits)])
else:
array_name_base = "_".join(["test", key_type, str(bits)])
array_name_priv = array_name_base + "_priv"
array_name_pub = array_name_base + "_pub"
# Convert bytearray to C array
c_array_priv = convert_der_to_c(array_name_priv, ASYMMETRIC_KEY_DATA[priv_key][bits])
c_array_pub = convert_der_to_c(array_name_pub, ASYMMETRIC_KEY_DATA[pub_key][bits])
# Write the C array to the output file
arrays.append(''.join(["\n", c_array_priv, "\n", c_array_pub, "\n"]))
# Update the lookup table
if key_type == "ec":
group_id_or_keybits = "MBEDTLS_ECP_DP_" + curve.upper()
else:
group_id_or_keybits = str(bits)
look_up_table.append(''.join(get_look_up_table_entry(key_type, group_id_or_keybits,
array_name_priv, array_name_pub)))
return ''.join(arrays), '\n'.join(look_up_table)
def main() -> None:
"""Command line entry point."""
default_output_path = build_tree.guess_project_root() + "/tests/include/test/test_keys.h"
argparser = argparse.ArgumentParser()
argparser.add_argument("--output", help="Output file", default=default_output_path)
args = argparser.parse_args()
output_file = args.output
# Support for 224 bit EC curves (secp224r1 and secp224k1) was removed from
# tf-psa-crypto. It only remains available for 3.6 LTS branch.
if not build_tree.is_mbedtls_3_6():
del EC_NAME_CONVERSION['PSA_ECC_FAMILY_SECP_R1'][224]
del EC_NAME_CONVERSION['PSA_ECC_FAMILY_SECP_R1'][192]
del EC_NAME_CONVERSION['PSA_ECC_FAMILY_SECP_K1'][192]
arrays, look_up_table = collect_keys()
write_output_file(output_file, arrays, look_up_table)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,649 @@
#!/usr/bin/env python3
# generate_tls13_compat_tests.py
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
"""
Generate TLSv1.3 Compat test cases
"""
import sys
import os
import argparse
import itertools
from collections import namedtuple
# define certificates configuration entry
Certificate = namedtuple("Certificate", ['cafile', 'certfile', 'keyfile'])
# define the certificate parameters for signature algorithms
CERTIFICATES = {
'ecdsa_secp256r1_sha256': Certificate('$DATA_FILES_PATH/test-ca2.crt',
'$DATA_FILES_PATH/ecdsa_secp256r1.crt',
'$DATA_FILES_PATH/ecdsa_secp256r1.key'),
'ecdsa_secp384r1_sha384': Certificate('$DATA_FILES_PATH/test-ca2.crt',
'$DATA_FILES_PATH/ecdsa_secp384r1.crt',
'$DATA_FILES_PATH/ecdsa_secp384r1.key'),
'ecdsa_secp521r1_sha512': Certificate('$DATA_FILES_PATH/test-ca2.crt',
'$DATA_FILES_PATH/ecdsa_secp521r1.crt',
'$DATA_FILES_PATH/ecdsa_secp521r1.key'),
'rsa_pss_rsae_sha256': Certificate('$DATA_FILES_PATH/test-ca_cat12.crt',
'$DATA_FILES_PATH/server2-sha256.crt',
'$DATA_FILES_PATH/server2.key')
}
CIPHER_SUITE_IANA_VALUE = {
"TLS_AES_128_GCM_SHA256": 0x1301,
"TLS_AES_256_GCM_SHA384": 0x1302,
"TLS_CHACHA20_POLY1305_SHA256": 0x1303,
"TLS_AES_128_CCM_SHA256": 0x1304,
"TLS_AES_128_CCM_8_SHA256": 0x1305
}
SIG_ALG_IANA_VALUE = {
"ecdsa_secp256r1_sha256": 0x0403,
"ecdsa_secp384r1_sha384": 0x0503,
"ecdsa_secp521r1_sha512": 0x0603,
'rsa_pss_rsae_sha256': 0x0804,
}
NAMED_GROUP_IANA_VALUE = {
'secp256r1': 0x17,
'secp384r1': 0x18,
'secp521r1': 0x19,
'x25519': 0x1d,
'x448': 0x1e,
# Only one finite field group to keep testing time within reasonable bounds.
'ffdhe2048': 0x100,
}
class TLSProgram:
"""
Base class for generate server/client command.
"""
# pylint: disable=too-many-arguments
def __init__(self, ciphersuite=None, signature_algorithm=None, named_group=None,
cert_sig_alg=None):
self._ciphers = []
self._sig_algs = []
self._named_groups = []
self._cert_sig_algs = []
if ciphersuite:
self.add_ciphersuites(ciphersuite)
if named_group:
self.add_named_groups(named_group)
if signature_algorithm:
self.add_signature_algorithms(signature_algorithm)
if cert_sig_alg:
self.add_cert_signature_algorithms(cert_sig_alg)
# add_ciphersuites should not override by sub class
def add_ciphersuites(self, *ciphersuites):
self._ciphers.extend(
[cipher for cipher in ciphersuites if cipher not in self._ciphers])
# add_signature_algorithms should not override by sub class
def add_signature_algorithms(self, *signature_algorithms):
self._sig_algs.extend(
[sig_alg for sig_alg in signature_algorithms if sig_alg not in self._sig_algs])
# add_named_groups should not override by sub class
def add_named_groups(self, *named_groups):
self._named_groups.extend(
[named_group for named_group in named_groups if named_group not in self._named_groups])
# add_cert_signature_algorithms should not override by sub class
def add_cert_signature_algorithms(self, *signature_algorithms):
self._cert_sig_algs.extend(
[sig_alg for sig_alg in signature_algorithms if sig_alg not in self._cert_sig_algs])
# pylint: disable=no-self-use
def pre_checks(self):
return []
# pylint: disable=no-self-use
def cmd(self):
if not self._cert_sig_algs:
self._cert_sig_algs = list(CERTIFICATES.keys())
return self.pre_cmd()
# pylint: disable=no-self-use
def post_checks(self):
return []
# pylint: disable=no-self-use
def pre_cmd(self):
return ['false']
# pylint: disable=unused-argument,no-self-use
def hrr_post_checks(self, named_group):
return []
class OpenSSLBase(TLSProgram):
"""
Generate base test commands for OpenSSL.
"""
NAMED_GROUP = {
'secp256r1': 'P-256',
'secp384r1': 'P-384',
'secp521r1': 'P-521',
'x25519': 'X25519',
'x448': 'X448',
'ffdhe2048': 'ffdhe2048',
}
def cmd(self):
ret = super().cmd()
if self._ciphers:
ciphersuites = ':'.join(self._ciphers)
ret += ["-ciphersuites {ciphersuites}".format(ciphersuites=ciphersuites)]
if self._sig_algs:
signature_algorithms = set(self._sig_algs + self._cert_sig_algs)
signature_algorithms = ':'.join(signature_algorithms)
ret += ["-sigalgs {signature_algorithms}".format(
signature_algorithms=signature_algorithms)]
if self._named_groups:
named_groups = ':'.join(
map(lambda named_group: self.NAMED_GROUP[named_group], self._named_groups))
ret += ["-groups {named_groups}".format(named_groups=named_groups)]
ret += ['-msg -tls1_3']
return ret
def pre_checks(self):
ret = ["requires_openssl_tls1_3"]
# ffdh groups require at least openssl 3.0
ffdh_groups = ['ffdhe2048']
if any(x in ffdh_groups for x in self._named_groups):
ret = ["requires_openssl_tls1_3_with_ffdh"]
return ret
class OpenSSLServ(OpenSSLBase):
"""
Generate test commands for OpenSSL server.
"""
def cmd(self):
ret = super().cmd()
ret += ['-num_tickets 0 -no_resume_ephemeral -no_cache']
return ret
def post_checks(self):
return ['-c "HTTP/1.0 200 ok"']
def pre_cmd(self):
ret = ['$O_NEXT_SRV_NO_CERT']
for _, cert, key in map(lambda sig_alg: CERTIFICATES[sig_alg], self._cert_sig_algs):
ret += ['-cert {cert} -key {key}'.format(cert=cert, key=key)]
return ret
class OpenSSLCli(OpenSSLBase):
"""
Generate test commands for OpenSSL client.
"""
def pre_cmd(self):
return ['$O_NEXT_CLI_NO_CERT',
'-CAfile {cafile}'.format(cafile=CERTIFICATES[self._cert_sig_algs[0]].cafile)]
class GnuTLSBase(TLSProgram):
"""
Generate base test commands for GnuTLS.
"""
CIPHER_SUITE = {
'TLS_AES_256_GCM_SHA384': [
'AES-256-GCM',
'SHA384',
'AEAD'],
'TLS_AES_128_GCM_SHA256': [
'AES-128-GCM',
'SHA256',
'AEAD'],
'TLS_CHACHA20_POLY1305_SHA256': [
'CHACHA20-POLY1305',
'SHA256',
'AEAD'],
'TLS_AES_128_CCM_SHA256': [
'AES-128-CCM',
'SHA256',
'AEAD'],
'TLS_AES_128_CCM_8_SHA256': [
'AES-128-CCM-8',
'SHA256',
'AEAD']}
SIGNATURE_ALGORITHM = {
'ecdsa_secp256r1_sha256': ['SIGN-ECDSA-SECP256R1-SHA256'],
'ecdsa_secp521r1_sha512': ['SIGN-ECDSA-SECP521R1-SHA512'],
'ecdsa_secp384r1_sha384': ['SIGN-ECDSA-SECP384R1-SHA384'],
'rsa_pss_rsae_sha256': ['SIGN-RSA-PSS-RSAE-SHA256']}
NAMED_GROUP = {
'secp256r1': ['GROUP-SECP256R1'],
'secp384r1': ['GROUP-SECP384R1'],
'secp521r1': ['GROUP-SECP521R1'],
'x25519': ['GROUP-X25519'],
'x448': ['GROUP-X448'],
'ffdhe2048': ['GROUP-FFDHE2048'],
}
def pre_checks(self):
return ["requires_gnutls_tls1_3",
"requires_gnutls_next_no_ticket"]
def cmd(self):
ret = super().cmd()
priority_string_list = []
def update_priority_string_list(items, map_table):
for item in items:
for i in map_table[item]:
if i not in priority_string_list:
yield i
if self._ciphers:
priority_string_list.extend(update_priority_string_list(
self._ciphers, self.CIPHER_SUITE))
else:
priority_string_list.extend(['CIPHER-ALL', 'MAC-ALL'])
if self._sig_algs:
signature_algorithms = set(self._sig_algs + self._cert_sig_algs)
priority_string_list.extend(update_priority_string_list(
signature_algorithms, self.SIGNATURE_ALGORITHM))
else:
priority_string_list.append('SIGN-ALL')
if self._named_groups:
priority_string_list.extend(update_priority_string_list(
self._named_groups, self.NAMED_GROUP))
else:
priority_string_list.append('GROUP-ALL')
priority_string_list = ['NONE'] + \
priority_string_list + ['VERS-TLS1.3']
priority_string = ':+'.join(priority_string_list)
priority_string += ':%NO_TICKETS'
ret += ['--priority={priority_string}'.format(
priority_string=priority_string)]
return ret
class GnuTLSServ(GnuTLSBase):
"""
Generate test commands for GnuTLS server.
"""
def pre_cmd(self):
ret = ['$G_NEXT_SRV_NO_CERT', '--http', '--disable-client-cert', '--debug=4']
for _, cert, key in map(lambda sig_alg: CERTIFICATES[sig_alg], self._cert_sig_algs):
ret += ['--x509certfile {cert} --x509keyfile {key}'.format(
cert=cert, key=key)]
return ret
def post_checks(self):
return ['-c "HTTP/1.0 200 OK"']
class GnuTLSCli(GnuTLSBase):
"""
Generate test commands for GnuTLS client.
"""
def pre_cmd(self):
return ['$G_NEXT_CLI_NO_CERT', '--debug=4', '--single-key-share',
'--x509cafile {cafile}'.format(cafile=CERTIFICATES[self._cert_sig_algs[0]].cafile)]
class MbedTLSBase(TLSProgram):
"""
Generate base test commands for mbedTLS.
"""
CIPHER_SUITE = {
'TLS_AES_256_GCM_SHA384': 'TLS1-3-AES-256-GCM-SHA384',
'TLS_AES_128_GCM_SHA256': 'TLS1-3-AES-128-GCM-SHA256',
'TLS_CHACHA20_POLY1305_SHA256': 'TLS1-3-CHACHA20-POLY1305-SHA256',
'TLS_AES_128_CCM_SHA256': 'TLS1-3-AES-128-CCM-SHA256',
'TLS_AES_128_CCM_8_SHA256': 'TLS1-3-AES-128-CCM-8-SHA256'}
def cmd(self):
ret = super().cmd()
ret += ['debug_level=4']
if self._ciphers:
ciphers = ','.join(
map(lambda cipher: self.CIPHER_SUITE[cipher], self._ciphers))
ret += ["force_ciphersuite={ciphers}".format(ciphers=ciphers)]
if self._sig_algs + self._cert_sig_algs:
ret += ['sig_algs={sig_algs}'.format(
sig_algs=','.join(set(self._sig_algs + self._cert_sig_algs)))]
if self._named_groups:
named_groups = ','.join(self._named_groups)
ret += ["groups={named_groups}".format(named_groups=named_groups)]
return ret
#pylint: disable=missing-function-docstring
def add_ffdh_group_requirements(self, requirement_list):
if 'ffdhe2048' in self._named_groups:
requirement_list.append('requires_config_enabled PSA_WANT_DH_RFC7919_2048')
if 'ffdhe3072' in self._named_groups:
requirement_list.append('requires_config_enabled PSA_WANT_DH_RFC7919_2048')
if 'ffdhe4096' in self._named_groups:
requirement_list.append('requires_config_enabled PSA_WANT_DH_RFC7919_2048')
if 'ffdhe6144' in self._named_groups:
requirement_list.append('requires_config_enabled PSA_WANT_DH_RFC7919_2048')
if 'ffdhe8192' in self._named_groups:
requirement_list.append('requires_config_enabled PSA_WANT_DH_RFC7919_2048')
def pre_checks(self):
ret = ['requires_config_enabled MBEDTLS_DEBUG_C',
'requires_config_enabled MBEDTLS_SSL_TLS1_3_KEY_EXCHANGE_MODE_EPHEMERAL_ENABLED']
if 'rsa_pss_rsae_sha256' in self._sig_algs + self._cert_sig_algs:
ret.append(
'requires_config_enabled MBEDTLS_X509_RSASSA_PSS_SUPPORT')
ec_groups = ['secp256r1', 'secp384r1', 'secp521r1', 'x25519', 'x448']
ffdh_groups = ['ffdhe2048', 'ffdhe3072', 'ffdhe4096', 'ffdhe6144', 'ffdhe8192']
if any(x in ec_groups for x in self._named_groups):
ret.append('requires_config_enabled PSA_WANT_ALG_ECDH')
if any(x in ffdh_groups for x in self._named_groups):
ret.append('requires_config_enabled PSA_WANT_ALG_FFDH')
self.add_ffdh_group_requirements(ret)
return ret
class MbedTLSServ(MbedTLSBase):
"""
Generate test commands for mbedTLS server.
"""
def cmd(self):
ret = super().cmd()
ret += ['tls13_kex_modes=ephemeral cookies=0 tickets=0']
return ret
def pre_checks(self):
return ['requires_config_enabled MBEDTLS_SSL_SRV_C'] + super().pre_checks()
def post_checks(self):
check_strings = ["Protocol is TLSv1.3"]
if self._ciphers:
check_strings.append(
"server hello, chosen ciphersuite: {} ( id={:04d} )".format(
self.CIPHER_SUITE[self._ciphers[0]],
CIPHER_SUITE_IANA_VALUE[self._ciphers[0]]))
if self._sig_algs:
check_strings.append(
"received signature algorithm: 0x{:x}".format(
SIG_ALG_IANA_VALUE[self._sig_algs[0]]))
for named_group in self._named_groups:
check_strings += ['got named group: {named_group}({iana_value:04x})'.format(
named_group=named_group,
iana_value=NAMED_GROUP_IANA_VALUE[named_group])]
check_strings.append("Certificate verification was skipped")
return ['-s "{}"'.format(i) for i in check_strings]
def pre_cmd(self):
ret = ['$P_SRV']
for _, cert, key in map(lambda sig_alg: CERTIFICATES[sig_alg], self._cert_sig_algs):
ret += ['crt_file={cert} key_file={key}'.format(cert=cert, key=key)]
return ret
def hrr_post_checks(self, named_group):
return ['-s "HRR selected_group: {:s}"'.format(named_group)]
class MbedTLSCli(MbedTLSBase):
"""
Generate test commands for mbedTLS client.
"""
def pre_cmd(self):
return ['$P_CLI',
'ca_file={cafile}'.format(cafile=CERTIFICATES[self._cert_sig_algs[0]].cafile)]
def pre_checks(self):
return ['requires_config_enabled MBEDTLS_SSL_CLI_C'] + super().pre_checks()
def hrr_post_checks(self, named_group):
ret = ['-c "received HelloRetryRequest message"']
ret += ['-c "selected_group ( {:d} )"'.format(NAMED_GROUP_IANA_VALUE[named_group])]
return ret
def post_checks(self):
check_strings = ["Protocol is TLSv1.3"]
if self._ciphers:
check_strings.append(
"server hello, chosen ciphersuite: ( {:04x} ) - {}".format(
CIPHER_SUITE_IANA_VALUE[self._ciphers[0]],
self.CIPHER_SUITE[self._ciphers[0]]))
if self._sig_algs:
check_strings.append(
"Certificate Verify: Signature algorithm ( {:04x} )".format(
SIG_ALG_IANA_VALUE[self._sig_algs[0]]))
for named_group in self._named_groups:
check_strings += ['NamedGroup: {named_group} ( {iana_value:x} )'.format(
named_group=named_group,
iana_value=NAMED_GROUP_IANA_VALUE[named_group])]
check_strings.append("Verifying peer X.509 certificate... ok")
return ['-c "{}"'.format(i) for i in check_strings]
SERVER_CLASSES = {'OpenSSL': OpenSSLServ, 'GnuTLS': GnuTLSServ, 'mbedTLS': MbedTLSServ}
CLIENT_CLASSES = {'OpenSSL': OpenSSLCli, 'GnuTLS': GnuTLSCli, 'mbedTLS': MbedTLSCli}
def generate_compat_test(client=None, server=None, cipher=None, named_group=None, sig_alg=None):
"""
Generate test case with `ssl-opt.sh` format.
"""
name = 'TLS 1.3 {client[0]}->{server[0]}: {cipher},{named_group},{sig_alg}'.format(
client=client, server=server, cipher=cipher[4:], sig_alg=sig_alg, named_group=named_group)
server_object = SERVER_CLASSES[server](ciphersuite=cipher,
named_group=named_group,
signature_algorithm=sig_alg,
cert_sig_alg=sig_alg)
client_object = CLIENT_CLASSES[client](ciphersuite=cipher,
named_group=named_group,
signature_algorithm=sig_alg,
cert_sig_alg=sig_alg)
cmd = ['run_test "{}"'.format(name),
'"{}"'.format(' '.join(server_object.cmd())),
'"{}"'.format(' '.join(client_object.cmd())),
'0']
cmd += server_object.post_checks()
cmd += client_object.post_checks()
cmd += ['-C "received HelloRetryRequest message"']
prefix = ' \\\n' + (' '*9)
cmd = prefix.join(cmd)
return '\n'.join(server_object.pre_checks() + client_object.pre_checks() + [cmd])
def generate_hrr_compat_test(client=None, server=None,
client_named_group=None, server_named_group=None,
cert_sig_alg=None):
"""
Generate Hello Retry Request test case with `ssl-opt.sh` format.
"""
name = 'TLS 1.3 {client[0]}->{server[0]}: HRR {c_named_group} -> {s_named_group}'.format(
client=client, server=server, c_named_group=client_named_group,
s_named_group=server_named_group)
server_object = SERVER_CLASSES[server](named_group=server_named_group,
cert_sig_alg=cert_sig_alg)
client_object = CLIENT_CLASSES[client](named_group=client_named_group,
cert_sig_alg=cert_sig_alg)
client_object.add_named_groups(server_named_group)
cmd = ['run_test "{}"'.format(name),
'"{}"'.format(' '.join(server_object.cmd())),
'"{}"'.format(' '.join(client_object.cmd())),
'0']
cmd += server_object.post_checks()
cmd += client_object.post_checks()
cmd += server_object.hrr_post_checks(server_named_group)
cmd += client_object.hrr_post_checks(server_named_group)
prefix = ' \\\n' + (' '*9)
cmd = prefix.join(cmd)
return '\n'.join(server_object.pre_checks() +
client_object.pre_checks() +
[cmd])
SSL_OUTPUT_HEADER = '''\
# TLS 1.3 interoperability test cases (equivalent of compat.sh for TLS 1.3).
#
# Automatically generated by {cmd}. Do not edit!
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
'''
DATA_FILES_PATH_VAR = '''
DATA_FILES_PATH=../framework/data_files
'''
def main():
"""
Main function of this program
"""
parser = argparse.ArgumentParser()
parser.add_argument('-o', '--output',
default='tests/opt-testcases/tls13-compat.sh',
help='Output file path (not used with -1)')
parser.add_argument('-1', '--single', action='store_true',
help='Print a single test case')
# Single mode used to be the default.
parser.add_argument('-a', '--generate-all-tls13-compat-tests',
action='store_false', dest='single',
help='Generate all test cases (negates -1) (default)')
parser.add_argument('--list-ciphers', action='store_true',
default=False, help='List supported ciphersuites')
parser.add_argument('--list-sig-algs', action='store_true',
default=False, help='List supported signature algorithms')
parser.add_argument('--list-named-groups', action='store_true',
default=False, help='List supported named groups')
parser.add_argument('--list-servers', action='store_true',
default=False, help='List supported TLS servers')
parser.add_argument('--list-clients', action='store_true',
default=False, help='List supported TLS Clients')
parser.add_argument('server', choices=SERVER_CLASSES.keys(), nargs='?',
default=list(SERVER_CLASSES.keys())[0],
help='Choose TLS server program for test')
parser.add_argument('client', choices=CLIENT_CLASSES.keys(), nargs='?',
default=list(CLIENT_CLASSES.keys())[0],
help='Choose TLS client program for test')
parser.add_argument('cipher', choices=CIPHER_SUITE_IANA_VALUE.keys(), nargs='?',
default=list(CIPHER_SUITE_IANA_VALUE.keys())[0],
help='Choose cipher suite for test')
parser.add_argument('sig_alg', choices=SIG_ALG_IANA_VALUE.keys(), nargs='?',
default=list(SIG_ALG_IANA_VALUE.keys())[0],
help='Choose cipher suite for test')
parser.add_argument('named_group', choices=NAMED_GROUP_IANA_VALUE.keys(), nargs='?',
default=list(NAMED_GROUP_IANA_VALUE.keys())[0],
help='Choose cipher suite for test')
args = parser.parse_args()
def get_all_test_cases():
# Generate normal compat test cases
for client, server, cipher, named_group, sig_alg in \
itertools.product(CLIENT_CLASSES.keys(),
SERVER_CLASSES.keys(),
CIPHER_SUITE_IANA_VALUE.keys(),
NAMED_GROUP_IANA_VALUE.keys(),
SIG_ALG_IANA_VALUE.keys()):
if server == 'mbedTLS' or client == 'mbedTLS':
yield generate_compat_test(client=client, server=server,
cipher=cipher, named_group=named_group,
sig_alg=sig_alg)
# Generate Hello Retry Request compat test cases
for client, server, client_named_group, server_named_group in \
itertools.product(CLIENT_CLASSES.keys(),
SERVER_CLASSES.keys(),
NAMED_GROUP_IANA_VALUE.keys(),
NAMED_GROUP_IANA_VALUE.keys()):
if (client == 'mbedTLS' or server == 'mbedTLS') and \
client_named_group != server_named_group:
yield generate_hrr_compat_test(client=client, server=server,
client_named_group=client_named_group,
server_named_group=server_named_group,
cert_sig_alg="ecdsa_secp256r1_sha256")
if not args.single:
if args.output:
with open(args.output, 'w', encoding="utf-8") as f:
f.write(SSL_OUTPUT_HEADER.format(
filename=os.path.basename(args.output),
cmd=os.path.basename(sys.argv[0])))
f.write(DATA_FILES_PATH_VAR)
f.write('\n\n'.join(get_all_test_cases()))
f.write('\n')
else:
print('\n\n'.join(get_all_test_cases()))
return 0
if args.list_ciphers or args.list_sig_algs or args.list_named_groups \
or args.list_servers or args.list_clients:
if args.list_ciphers:
print(*CIPHER_SUITE_IANA_VALUE.keys())
if args.list_sig_algs:
print(*SIG_ALG_IANA_VALUE.keys())
if args.list_named_groups:
print(*NAMED_GROUP_IANA_VALUE.keys())
if args.list_servers:
print(*SERVER_CLASSES.keys())
if args.list_clients:
print(*CLIENT_CLASSES.keys())
return 0
print(generate_compat_test(server=args.server, client=args.client, sig_alg=args.sig_alg,
cipher=args.cipher, named_group=args.named_group))
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,16 @@
#!/usr/bin/env python3
"""
Generate miscellaneous TLS test cases relating to the handshake.
Transitional wrapper to facilitate the migration of consuming branches.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import sys
from mbedtls_framework import tls_handshake_tests
if __name__ == '__main__':
sys.argv[1:1] = ["--no-tls12-client-hello-defragmentation-support"]
tls_handshake_tests.main()

103
vendor/mbedtls/framework/scripts/lcov.sh vendored Executable file
View File

@@ -0,0 +1,103 @@
#!/bin/sh
help () {
cat <<EOF
Usage: $0 [-r]
Collect coverage statistics of library code into an HTML report.
General instructions:
1. Build the library with CFLAGS="--coverage -O0 -g3" and link the test
programs with LDFLAGS="--coverage".
This can be an out-of-tree build.
For example (in-tree):
cmake -D CMAKE_BUILD_TYPE=Coverage . && make
Or (out-of-tree):
mkdir build-coverage && cd build-coverage &&
cmake -D CMAKE_BUILD_TYPE=Coverage .. && make
2. Run whatever tests you want.
3. Run this script from the parent of the directory containing the library
object files and coverage statistics files.
4. Browse the coverage report in Coverage/index.html.
5. After rework, run "$0 -r", then re-test and run "$0" to get a fresh report.
Options
-r Reset traces. Run this before re-testing to get fresh measurements.
EOF
}
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
# This script must be invoked from the project's root.
set -eu
scriptdir=$(dirname "$(realpath $0)")
. $scriptdir/project_detection.sh
# Collect stats and build a HTML report.
lcov_library_report () {
rm -rf Coverage
mkdir Coverage Coverage/tmp
# Pass absolute paths as lcov output files. This works around a bug
# whereby lcov tries to create the output file in the root directory
# if it has emitted a warning. A fix was released in lcov 1.13 in 2016.
# Ubuntu 16.04 is affected, 18.04 and above are not.
# https://github.com/linux-test-project/lcov/commit/632c25a0d1f5e4d2f4fd5b28ce7c8b86d388c91f
COVTMP=$PWD/Coverage/tmp
lcov --capture --initial ${lcov_dirs} -o "$COVTMP/files.info"
lcov --rc lcov_branch_coverage=1 --capture ${lcov_dirs} -o "$COVTMP/tests.info"
lcov --rc lcov_branch_coverage=1 --add-tracefile "$COVTMP/files.info" --add-tracefile "$COVTMP/tests.info" -o "$COVTMP/all.info"
lcov --rc lcov_branch_coverage=1 --remove "$COVTMP/all.info" -o "$COVTMP/final.info" '*.h'
gendesc tests/Descriptions.txt -o "$COVTMP/descriptions"
genhtml --title "$title" --description-file "$COVTMP/descriptions" --keep-descriptions --legend --branch-coverage -o Coverage "$COVTMP/final.info"
rm -f "$COVTMP/"*.info "$COVTMP/descriptions"
echo "Coverage report in: Coverage/index.html"
}
# Reset the traces to 0.
lcov_reset_traces () {
# Location with plain make
for dir in ${library_dirs}; do
rm -f ${dir}/*.gcda
done
# Location with CMake
for dir in ${library_dirs}; do
rm -f ${dir}/CMakeFiles/*.dir/*.gcda
done
}
if [ $# -gt 0 ] && [ "$1" = "--help" ]; then
help
exit
fi
if [ -d tf-psa-crypto ]; then
library_dirs='library tf-psa-crypto/core tf-psa-crypto/drivers/builtin'
title='Mbed TLS >= 4.0'
elif in_mbedtls_repo; then
library_dirs='library'
title='Mbed TLS < 4.0'
elif in_tf_psa_crypto_repo; then
library_dirs='core drivers/builtin'
title='TF-PSA-Crypto'
else
echo "Cannot auto detect build files"
exit
fi
lcov_dirs=""
for dir in ${library_dirs}; do
lcov_dirs="${lcov_dirs} --directory ${dir}"
done
main=lcov_library_report
while getopts r OPTLET; do
case $OPTLET in
r) main=lcov_reset_traces;;
*) help 2>&1; exit 120;;
esac
done
shift $((OPTIND - 1))
"$main" "$@"

View File

@@ -0,0 +1,140 @@
#!/usr/bin/env python3
"""Generate, check and list the generated files
Transitional wrapper to facilitate the migration of consuming branches.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import sys
from pathlib import Path
from mbedtls_framework import build_tree
from mbedtls_framework import generated_files
from mbedtls_framework.generated_files import GenerationScript, get_generation_script_files
COMMON_GENERATION_SCRIPTS = [
GenerationScript(
Path("scripts/generate_config_checks.py"),
get_generation_script_files("scripts/generate_config_checks.py"),
output_dir_option="",
optional=True)
]
if build_tree.looks_like_tf_psa_crypto_root("."):
TF_PSA_CRYPTO_GENERATION_SCRIPTS = [
GenerationScript(
Path("scripts/generate_driver_wrappers.py"),
[Path("core/psa_crypto_driver_wrappers.h"),
Path("core/psa_crypto_driver_wrappers_no_static.c")],
"", None
),
GenerationScript(
Path("framework/scripts/generate_test_keys.py"),
[Path("tests/include/test/test_keys.h")],
None, "--output"
),
GenerationScript(
Path("scripts/generate_psa_constants.py"),
[Path("programs/psa/psa_constant_names_generated.c")],
"", None
),
GenerationScript(
Path("framework/scripts/generate_bignum_tests.py"),
get_generation_script_files("framework/scripts/generate_bignum_tests.py"),
"--directory", None
),
GenerationScript(
Path("framework/scripts/generate_config_tests.py"),
get_generation_script_files("framework/scripts/generate_config_tests.py"),
"--directory", None
),
GenerationScript(
Path("framework/scripts/generate_ecp_tests.py"),
get_generation_script_files("framework/scripts/generate_ecp_tests.py"),
"--directory", None
),
GenerationScript(
Path("framework/scripts/generate_psa_tests.py"),
get_generation_script_files("framework/scripts/generate_psa_tests.py"),
"--directory", None
),
]
if build_tree.looks_like_mbedtls_root(".") and not build_tree.is_mbedtls_3_6():
MBEDTLS_GENERATION_SCRIPTS = [
GenerationScript(
Path("scripts/generate_errors.pl"),
[Path("library/error.c")],
None, "tf-psa-crypto/drivers/builtin/include/mbedtls \
include/mbedtls/ \
scripts/data_files"
),
GenerationScript(
Path("scripts/generate_features.pl"),
[Path("library/version_features.c")],
None, "include/mbedtls/ scripts/data_files"
),
GenerationScript(
Path("framework/scripts/generate_ssl_debug_helpers.py"),
[Path("library/ssl_debug_helpers_generated.c")],
"", None
),
GenerationScript(
Path("framework/scripts/generate_test_keys.py"),
[Path("tests/include/test/test_keys.h")],
None, "--output"
),
GenerationScript(
Path("framework/scripts/generate_test_cert_macros.py"),
[Path("tests/include/test/test_certs.h")],
None, "--output"
),
GenerationScript(
Path("scripts/generate_query_config.pl"),
[Path("programs/test/query_config.c")],
None, "include/mbedtls/mbedtls_config.h \
tf-psa-crypto/include/psa/crypto_config.h \
scripts/data_files/query_config.fmt"
),
GenerationScript(
Path("framework/scripts/generate_config_tests.py"),
get_generation_script_files("framework/scripts/generate_config_tests.py"),
"--directory", None
),
GenerationScript(
Path("framework/scripts/generate_tls13_compat_tests.py"),
[Path("tests/opt-testcases/tls13-compat.sh")],
None, "--output"
),
GenerationScript(
Path("framework/scripts/generate_tls_handshake_tests.py"),
[Path("tests/opt-testcases/handshake-generated.sh")],
None, "--output"
),
]
if Path("scripts/generate_visualc_files.pl").is_file():
MBEDTLS_GENERATION_SCRIPTS.append(
GenerationScript(
Path("scripts/generate_visualc_files.pl"),
get_generation_script_files("scripts/generate_visualc_files.pl"),
"--directory", None))
def main() -> int:
if not build_tree.looks_like_root("."):
raise RuntimeError("This script must be run from Mbed TLS or TF-PSA-Crypto root.")
if build_tree.looks_like_tf_psa_crypto_root("."):
generation_scripts = TF_PSA_CRYPTO_GENERATION_SCRIPTS
elif not build_tree.is_mbedtls_3_6():
generation_scripts = MBEDTLS_GENERATION_SCRIPTS
else:
raise Exception("No support for Mbed TLS 3.6")
generation_scripts += COMMON_GENERATION_SCRIPTS
return generated_files.main(generation_scripts)
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,36 @@
#!/usr/bin/env perl
# Parse a massif.out.xxx file and output peak total memory usage
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use warnings;
use strict;
use utf8;
use open qw(:std utf8);
die unless @ARGV == 1;
my @snaps;
open my $fh, '<', $ARGV[0] or die;
{ local $/ = 'snapshot='; @snaps = <$fh>; }
close $fh or die;
my ($max, $max_heap, $max_he, $max_stack) = (0, 0, 0, 0);
for (@snaps)
{
my ($heap, $heap_extra, $stack) = m{
mem_heap_B=(\d+)\n
mem_heap_extra_B=(\d+)\n
mem_stacks_B=(\d+)
}xm;
next unless defined $heap;
my $total = $heap + $heap_extra + $stack;
if( $total > $max ) {
($max, $max_heap, $max_he, $max_stack) = ($total, $heap, $heap_extra, $stack);
}
}
printf "$max (heap $max_heap+$max_he, stack $max_stack)\n";

View File

@@ -0,0 +1,3 @@
# This file needs to exist to make mbedtls_framework a package.
# Among other things, this allows modules in this directory to make
# relative imports.

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,416 @@
"""Common features for bignum in test generation framework."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
from abc import abstractmethod
import enum
from typing import Iterator, List, Tuple, TypeVar, Any
from copy import deepcopy
from itertools import chain
from math import ceil
from . import test_case
from . import test_data_generation
from .bignum_data import INPUTS_DEFAULT, MODULI_DEFAULT
T = TypeVar('T') #pylint: disable=invalid-name
def invmod(a: int, n: int) -> int:
"""Return inverse of a to modulo n.
Warning: the output might be negative! See invmod_positive().
"""
b, c = 1, 0
while n:
q, r = divmod(a, n)
a, b, c, n = n, c, b - q*c, r
# at this point a is the gcd of the original inputs
if a == 1:
return b
raise ValueError("Not invertible")
def invmod_positive(a: int, n: int) -> int:
"""Return a non-negative inverse of a to modulo n.
Equivalent to pow(a, -1, n) in Python 3.8+.
"""
inv = invmod(a, n)
return inv if inv >= 0 else inv + n
def hex_to_int(val: str) -> int:
"""Implement the syntax accepted by mbedtls_test_read_mpi().
This is a superset of what is accepted by mbedtls_test_read_mpi_core().
"""
if val in ['', '-']:
return 0
return int(val, 16)
def quote_str(val: str) -> str:
return "\"{}\"".format(val)
def bound_mpi(val: int, bits_in_limb: int) -> int:
"""First number exceeding number of limbs needed for given input value."""
return bound_mpi_limbs(limbs_mpi(val, bits_in_limb), bits_in_limb)
def bound_mpi_limbs(limbs: int, bits_in_limb: int) -> int:
"""First number exceeding maximum of given number of limbs."""
bits = bits_in_limb * limbs
return 1 << bits
def limbs_mpi(val: int, bits_in_limb: int) -> int:
"""Return the number of limbs required to store value."""
bit_length = max(val.bit_length(), 1)
return (bit_length + bits_in_limb - 1) // bits_in_limb
def combination_pairs(values: List[T]) -> List[Tuple[T, T]]:
"""Return all pair combinations from input values."""
return [(x, y) for x in values for y in values]
def combination_two_lists(first_vals: List[T], second_vals: List[T]) -> List[Tuple[T, T]]:
"""Return all pair combinations from two input lists"""
return [(x, y) for x in first_vals for y in second_vals]
def expand_list_negative(values: List[str]) -> List[str]:
"""Adds the negative of every element in the list to the list"""
return values + [f"-{value}" for value in values]
def bits_to_limbs(bits: int, bits_in_limb: int) -> int:
""" Return the appropriate ammount of limbs needed to store
a number contained in input bits"""
return ceil(bits / bits_in_limb)
def hex_digits_for_limb(limbs: int, bits_in_limb: int) -> int:
""" Return the hex digits need for a number of limbs. """
return 2 * ((limbs * bits_in_limb) // 8)
def hex_digits_max_int(val: str, bits_in_limb: int) -> int:
""" Return the first number exceeding maximum the limb space
required to store the input hex-string value. This method
weights on the input str_len rather than numerical value
and works with zero-padded inputs"""
n = ((1 << (len(val) * 4)) - 1)
l = limbs_mpi(n, bits_in_limb)
return bound_mpi_limbs(l, bits_in_limb)
def zfill_match(reference: str, target: str) -> str:
""" Zero pad target hex-string to match the limb size of
the reference input """
lt = len(target)
lr = len(reference)
target_len = lr if lt < lr else lt
return "{:x}".format(int(target, 16)).zfill(target_len)
class OperationCommon(test_data_generation.BaseTest):
"""Common features for bignum binary operations.
This adds functionality common in binary operation tests.
Attributes:
symbol: Symbol to use for the operation in case description.
input_values: List of values to use as test case inputs. These are
combined to produce pairs of values.
input_cases: List of tuples containing pairs of test case inputs. This
can be used to implement specific pairs of inputs.
unique_combinations_only: Boolean to select if test case combinations
must be unique. If True, only A,B or B,A would be included as a test
case. If False, both A,B and B,A would be included.
input_style: Controls the way how test data is passed to the functions
in the generated test cases. "variable" passes them as they are
defined in the python source. "arch_split" pads the values with
zeroes depending on the architecture/limb size. If this is set,
test cases are generated for all architectures.
arity: the number of operands for the operation. Currently supported
values are 1 and 2.
"""
symbol = ""
input_values = INPUTS_DEFAULT # type: List[str]
input_cases = [] # type: List[Any]
dependencies = [] # type: List[Any]
unique_combinations_only = False
input_styles = ["variable", "fixed", "arch_split"] # type: List[str]
input_style = "variable" # type: str
limb_sizes = [32, 64] # type: List[int]
arities = [1, 2]
arity = 2
suffix = False # for arity = 1, symbol can be prefix (default) or suffix
def __init__(self, val_a: str, val_b: str = "0", bits_in_limb: int = 32) -> None:
self.val_a = val_a
self.val_b = val_b
# Setting the int versions here as opposed to making them @properties
# provides earlier/more robust input validation.
self.int_a = hex_to_int(val_a)
self.int_b = hex_to_int(val_b)
self.dependencies = deepcopy(self.dependencies)
if bits_in_limb not in self.limb_sizes:
raise ValueError("Invalid number of bits in limb!")
if self.input_style == "arch_split":
self.dependencies.append("MBEDTLS_HAVE_INT{:d}".format(bits_in_limb))
self.bits_in_limb = bits_in_limb
@property
def boundary(self) -> int:
if self.arity == 1:
return self.int_a
elif self.arity == 2:
return max(self.int_a, self.int_b)
raise ValueError("Unsupported number of operands!")
@property
def limb_boundary(self) -> int:
return bound_mpi(self.boundary, self.bits_in_limb)
@property
def limbs(self) -> int:
return limbs_mpi(self.boundary, self.bits_in_limb)
@property
def hex_digits(self) -> int:
return hex_digits_for_limb(self.limbs, self.bits_in_limb)
def format_arg(self, val: str) -> str:
if self.input_style not in self.input_styles:
raise ValueError("Unknown input style!")
if self.input_style == "variable":
return val
else:
return val.zfill(self.hex_digits)
def format_result(self, res: int) -> str:
res_str = '{:x}'.format(res)
return quote_str(self.format_arg(res_str))
@property
def arg_a(self) -> str:
return self.format_arg(self.val_a)
@property
def arg_b(self) -> str:
if self.arity == 1:
raise AttributeError("Operation is unary and doesn't have arg_b!")
return self.format_arg(self.val_b)
def arguments(self) -> List[str]:
args = [quote_str(self.arg_a)]
if self.arity == 2:
args.append(quote_str(self.arg_b))
return args + self.result()
def description(self) -> str:
"""Generate a description for the test case.
If not set, case_description uses the form A `symbol` B, where symbol
is used to represent the operation. Descriptions of each value are
generated to provide some context to the test case.
"""
if not self.case_description:
if self.arity == 1:
format_string = "{1:x} {0}" if self.suffix else "{0} {1:x}"
self.case_description = format_string.format(
self.symbol, self.int_a
)
elif self.arity == 2:
self.case_description = "{:x} {} {:x}".format(
self.int_a, self.symbol, self.int_b
)
return super().description()
@property
def is_valid(self) -> bool:
return True
@abstractmethod
def result(self) -> List[str]:
"""Get the result of the operation.
This could be calculated during initialization and stored as `_result`
and then returned, or calculated when the method is called.
"""
raise NotImplementedError
@classmethod
def get_value_pairs(cls) -> Iterator[Tuple[str, str]]:
"""Generator to yield pairs of inputs.
Combinations are first generated from all input values, and then
specific cases provided.
"""
if cls.arity == 1:
yield from ((a, "0") for a in cls.input_values)
elif cls.arity == 2:
if cls.unique_combinations_only:
yield from combination_pairs(cls.input_values)
else:
yield from (
(a, b)
for a in cls.input_values
for b in cls.input_values
)
else:
raise ValueError("Unsupported number of operands!")
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
if cls.input_style not in cls.input_styles:
raise ValueError("Unknown input style!")
if cls.arity not in cls.arities:
raise ValueError("Unsupported number of operands!")
if cls.input_style == "arch_split":
test_objects = (cls(a, b, bits_in_limb=bil)
for a, b in cls.get_value_pairs()
for bil in cls.limb_sizes)
special_cases = (cls(*args, bits_in_limb=bil) # type: ignore
for args in cls.input_cases
for bil in cls.limb_sizes)
else:
test_objects = (cls(a, b)
for a, b in cls.get_value_pairs())
special_cases = (cls(*args) for args in cls.input_cases)
yield from (valid_test_object.create_test_case()
for valid_test_object in filter(
lambda test_object: test_object.is_valid,
chain(test_objects, special_cases)
)
)
class ModulusRepresentation(enum.Enum):
"""Representation selector of a modulus."""
# Numerical values aligned with the type mbedtls_mpi_mod_rep_selector
INVALID = 0
MONTGOMERY = 2
OPT_RED = 3
def symbol(self) -> str:
"""The C symbol for this representation selector."""
return 'MBEDTLS_MPI_MOD_REP_' + self.name
@classmethod
def supported_representations(cls) -> List['ModulusRepresentation']:
"""Return all representations that are supported in positive test cases."""
return [cls.MONTGOMERY, cls.OPT_RED]
class ModOperationCommon(OperationCommon):
#pylint: disable=abstract-method
"""Target for bignum mod_raw test case generation."""
moduli = MODULI_DEFAULT # type: List[str]
montgomery_form_a = False
disallow_zero_a = False
def __init__(self, val_n: str, val_a: str, val_b: str = "0",
bits_in_limb: int = 64) -> None:
super().__init__(val_a=val_a, val_b=val_b, bits_in_limb=bits_in_limb)
self.val_n = val_n
# Setting the int versions here as opposed to making them @properties
# provides earlier/more robust input validation.
self.int_n = hex_to_int(val_n)
def to_montgomery(self, val: int) -> int:
return (val * self.r) % self.int_n
def from_montgomery(self, val: int) -> int:
return (val * self.r_inv) % self.int_n
def convert_from_canonical(self, canonical: int,
rep: ModulusRepresentation) -> int:
"""Convert values from canonical representation to the given representation."""
if rep is ModulusRepresentation.MONTGOMERY:
return self.to_montgomery(canonical)
elif rep is ModulusRepresentation.OPT_RED:
return canonical
else:
raise ValueError('Modulus representation not supported: {}'
.format(rep.name))
@property
def boundary(self) -> int:
return self.int_n
@property
def arg_a(self) -> str:
if self.montgomery_form_a:
value_a = self.to_montgomery(self.int_a)
else:
value_a = self.int_a
return self.format_arg('{:x}'.format(value_a))
@property
def arg_n(self) -> str:
return self.format_arg(self.val_n)
def format_arg(self, val: str) -> str:
return super().format_arg(val).zfill(self.hex_digits)
def arguments(self) -> List[str]:
return [quote_str(self.arg_n)] + super().arguments()
@property
def r(self) -> int: # pylint: disable=invalid-name
l = limbs_mpi(self.int_n, self.bits_in_limb)
return bound_mpi_limbs(l, self.bits_in_limb)
@property
def r_inv(self) -> int:
return invmod(self.r, self.int_n)
@property
def r2(self) -> int: # pylint: disable=invalid-name
return pow(self.r, 2)
@property
def is_valid(self) -> bool:
if self.int_a >= self.int_n:
return False
if self.disallow_zero_a and self.int_a == 0:
return False
if self.arity == 2 and self.int_b >= self.int_n:
return False
return True
def description(self) -> str:
"""Generate a description for the test case.
It uses the form A `symbol` B mod N, where symbol is used to represent
the operation.
"""
if not self.case_description:
return super().description() + " mod {:x}".format(self.int_n)
return super().description()
@classmethod
def input_cases_args(cls) -> Iterator[Tuple[Any, Any, Any]]:
if cls.arity == 1:
yield from ((n, a, "0") for a, n in cls.input_cases)
elif cls.arity == 2:
yield from ((n, a, b) for a, b, n in cls.input_cases)
else:
raise ValueError("Unsupported number of operands!")
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
if cls.input_style not in cls.input_styles:
raise ValueError("Unknown input style!")
if cls.arity not in cls.arities:
raise ValueError("Unsupported number of operands!")
if cls.input_style == "arch_split":
test_objects = (cls(n, a, b, bits_in_limb=bil)
for n in cls.moduli
for a, b in cls.get_value_pairs()
for bil in cls.limb_sizes)
special_cases = (cls(*args, bits_in_limb=bil)
for args in cls.input_cases_args()
for bil in cls.limb_sizes)
else:
test_objects = (cls(n, a, b)
for n in cls.moduli
for a, b in cls.get_value_pairs())
special_cases = (cls(*args) for args in cls.input_cases_args())
yield from (valid_test_object.create_test_case()
for valid_test_object in filter(
lambda test_object: test_object.is_valid,
chain(test_objects, special_cases)
))

View File

@@ -0,0 +1,972 @@
"""Framework classes for generation of bignum core test cases."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import math
import random
from typing import Dict, Iterator, List, Tuple
from . import test_case
from . import test_data_generation
from . import bignum_common
from . import bignum_data
class BignumCoreTarget(test_data_generation.BaseTarget):
#pylint: disable=abstract-method, too-few-public-methods
"""Target for bignum core test case generation."""
target_basename = 'test_suite_bignum_core.generated'
class BignumCoreShiftR(BignumCoreTarget, test_data_generation.BaseTest):
"""Test cases for mbedtls_bignum_core_shift_r()."""
count = 0
test_function = "mpi_core_shift_r"
test_name = "Core shift right"
DATA = [
('00', '0', [0, 1, 8]),
('01', '1', [0, 1, 2, 8, 64]),
('dee5ca1a7ef10a75', '64-bit',
list(range(11)) + [31, 32, 33, 63, 64, 65, 71, 72]),
('002e7ab0070ad57001', '[leading 0 limb]',
[0, 1, 8, 63, 64]),
('a1055eb0bb1efa1150ff', '80-bit',
[0, 1, 8, 63, 64, 65, 72, 79, 80, 81, 88, 128, 129, 136]),
('020100000000000000001011121314151617', '138-bit',
[0, 1, 8, 9, 16, 72, 73, 136, 137, 138, 144]),
]
def __init__(self, input_hex: str, descr: str, count: int) -> None:
self.input_hex = input_hex
self.number_description = descr
self.shift_count = count
self.result = bignum_common.hex_to_int(input_hex) >> count
def arguments(self) -> List[str]:
return ['"{}"'.format(self.input_hex),
str(self.shift_count),
'"{:0{}x}"'.format(self.result, len(self.input_hex))]
def description(self) -> str:
return 'Core shift {} >> {}'.format(self.number_description,
self.shift_count)
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
for input_hex, descr, counts in cls.DATA:
for count in counts:
yield cls(input_hex, descr, count).create_test_case()
class BignumCoreShiftL(BignumCoreTarget, bignum_common.ModOperationCommon):
"""Test cases for mbedtls_bignum_core_shift_l()."""
BIT_SHIFT_VALUES = ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'a',
'1f', '20', '21', '3f', '40', '41', '47', '48', '4f',
'50', '51', '58', '80', '81', '88']
DATA = ["0", "1", "40", "dee5ca1a7ef10a75", "a1055eb0bb1efa1150ff",
"002e7ab0070ad57001", "020100000000000000001011121314151617",
"1946e2958a85d8863ae21f4904fcc49478412534ed53eaf321f63f2a222" +
"7a3c63acbf50b6305595f90cfa8327f6db80d986fe96080bcbb5df1bdbe" +
"9b74fb8dedf2bddb3f8215b54dffd66409323bcc473e45a8fe9d08e77a51" +
"1698b5dad0416305db7fcf"]
arity = 1
test_function = "mpi_core_shift_l"
test_name = "Core shift(L)"
input_style = "arch_split"
symbol = "<<"
input_values = BIT_SHIFT_VALUES
moduli = DATA
@property
def val_n_max_limbs(self) -> int:
""" Return the limb count required to store the maximum number that can
fit in a the number of digits used by val_n """
m = bignum_common.hex_digits_max_int(self.val_n, self.bits_in_limb) - 1
return bignum_common.limbs_mpi(m, self.bits_in_limb)
def arguments(self) -> List[str]:
return [bignum_common.quote_str(self.val_n),
str(self.int_a)
] + self.result()
def description(self) -> str:
""" Format the output as:
#{count} {hex input} ({input bits} {limbs capacity}) << {bit shift} """
bits = "({} bits in {} limbs)".format(self.int_n.bit_length(), self.val_n_max_limbs)
return "{} #{} {} {} {} {}".format(self.test_name,
self.count,
self.val_n,
bits,
self.symbol,
self.int_a)
def format_result(self, res: int) -> str:
# Override to match zero-pading for leading digits between the output and input.
res_str = bignum_common.zfill_match(self.val_n, "{:x}".format(res))
return bignum_common.quote_str(res_str)
def result(self) -> List[str]:
result = (self.int_n << self.int_a)
# Calculate if there is space for shifting to the left(leading zero limbs)
mx = bignum_common.hex_digits_max_int(self.val_n, self.bits_in_limb)
# If there are empty limbs ahead, adjust the bitmask accordingly
result = result & (mx - 1)
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
class BignumCoreCTLookup(BignumCoreTarget, test_data_generation.BaseTest):
"""Test cases for mbedtls_mpi_core_ct_uint_table_lookup()."""
test_function = "mpi_core_ct_uint_table_lookup"
test_name = "Constant time MPI table lookup"
bitsizes = [
(32, "One limb"),
(192, "Smallest curve sized"),
(512, "Largest curve sized"),
(2048, "Small FF/RSA sized"),
(4096, "Large FF/RSA sized"),
]
window_sizes = [0, 1, 2, 3, 4, 5, 6]
def __init__(self,
bitsize: int, descr: str, window_size: int) -> None:
self.bitsize = bitsize
self.bitsize_description = descr
self.window_size = window_size
def arguments(self) -> List[str]:
return [str(self.bitsize), str(self.window_size)]
def description(self) -> str:
return '{} - {} MPI with {} bit window'.format(
BignumCoreCTLookup.test_name,
self.bitsize_description,
self.window_size
)
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
for bitsize, bitsize_description in cls.bitsizes:
for window_size in cls.window_sizes:
yield (cls(bitsize, bitsize_description, window_size)
.create_test_case())
class BignumCoreAddAndAddIf(BignumCoreTarget, bignum_common.OperationCommon):
"""Test cases for bignum core add and add-if."""
count = 0
symbol = "+"
test_function = "mpi_core_add_and_add_if"
test_name = "mpi_core_add_and_add_if"
input_style = "arch_split"
input_values = bignum_data.ADD_SUB_DATA
unique_combinations_only = True
def result(self) -> List[str]:
result = self.int_a + self.int_b
carry, result = divmod(result, self.limb_boundary)
return [
self.format_result(result),
str(carry)
]
class BignumCoreSub(BignumCoreTarget, bignum_common.OperationCommon):
"""Test cases for bignum core sub."""
count = 0
input_style = "arch_split"
symbol = "-"
test_function = "mpi_core_sub"
test_name = "mbedtls_mpi_core_sub"
input_values = bignum_data.ADD_SUB_DATA
def result(self) -> List[str]:
if self.int_a >= self.int_b:
result = self.int_a - self.int_b
carry = 0
else:
result = self.limb_boundary + self.int_a - self.int_b
carry = 1
return [
self.format_result(result),
str(carry)
]
class BignumCoreMLA(BignumCoreTarget, bignum_common.OperationCommon):
"""Test cases for fixed-size multiply accumulate."""
count = 0
test_function = "mpi_core_mla"
test_name = "mbedtls_mpi_core_mla"
input_values = [
"0", "1", "fffe", "ffffffff", "100000000", "20000000000000",
"ffffffffffffffff", "10000000000000000", "1234567890abcdef0",
"fffffffffffffffffefefefefefefefe",
"100000000000000000000000000000000",
"1234567890abcdef01234567890abcdef0",
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
"1234567890abcdef01234567890abcdef01234567890abcdef01234567890abcdef0",
(
"4df72d07b4b71c8dacb6cffa954f8d88254b6277099308baf003fab73227f" +
"34029643b5a263f66e0d3c3fa297ef71755efd53b8fb6cb812c6bbf7bcf17" +
"9298bd9947c4c8b14324140a2c0f5fad7958a69050a987a6096e9f055fb38" +
"edf0c5889eca4a0cfa99b45fbdeee4c696b328ddceae4723945901ec02507" +
"6b12b"
)
] # type: List[str]
input_scalars = [
"0", "3", "fe", "ff", "ffff", "10000", "ffffffff", "100000000",
"7f7f7f7f7f7f7f7f", "8000000000000000", "fffffffffffffffe"
] # type: List[str]
def __init__(self, val_a: str, val_b: str, val_s: str) -> None:
super().__init__(val_a, val_b)
self.arg_scalar = val_s
self.int_scalar = bignum_common.hex_to_int(val_s)
if bignum_common.limbs_mpi(self.int_scalar, 32) > 1:
self.dependencies = ["MBEDTLS_HAVE_INT64"]
def arguments(self) -> List[str]:
return [
bignum_common.quote_str(self.arg_a),
bignum_common.quote_str(self.arg_b),
bignum_common.quote_str(self.arg_scalar)
] + self.result()
def description(self) -> str:
"""Override and add the additional scalar."""
if not self.case_description:
self.case_description = "0x{} + 0x{} * 0x{}".format(
self.arg_a, self.arg_b, self.arg_scalar
)
return super().description()
def result(self) -> List[str]:
result = self.int_a + (self.int_b * self.int_scalar)
bound_val = max(self.int_a, self.int_b)
bound_4 = bignum_common.bound_mpi(bound_val, 32)
bound_8 = bignum_common.bound_mpi(bound_val, 64)
carry_4, remainder_4 = divmod(result, bound_4)
carry_8, remainder_8 = divmod(result, bound_8)
return [
"\"{:x}\"".format(remainder_4),
"\"{:x}\"".format(carry_4),
"\"{:x}\"".format(remainder_8),
"\"{:x}\"".format(carry_8)
]
@classmethod
def get_value_pairs(cls) -> Iterator[Tuple[str, str]]:
"""Generator to yield pairs of inputs.
Combinations are first generated from all input values, and then
specific cases provided.
"""
yield from super().get_value_pairs()
yield from cls.input_cases
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
"""Override for additional scalar input."""
for a_value, b_value in cls.get_value_pairs():
for s_value in cls.input_scalars:
cur_op = cls(a_value, b_value, s_value)
yield cur_op.create_test_case()
class BignumCoreMul(BignumCoreTarget, bignum_common.OperationCommon):
"""Test cases for bignum core multiplication."""
count = 0
input_style = "arch_split"
symbol = "*"
test_function = "mpi_core_mul"
test_name = "mbedtls_mpi_core_mul"
arity = 2
unique_combinations_only = True
def format_arg(self, val: str) -> str:
return val
def format_result(self, res: int) -> str:
res_str = '{:x}'.format(res)
a_limbs = bignum_common.limbs_mpi(self.int_a, self.bits_in_limb)
b_limbs = bignum_common.limbs_mpi(self.int_b, self.bits_in_limb)
hex_digits = bignum_common.hex_digits_for_limb(a_limbs + b_limbs, self.bits_in_limb)
return bignum_common.quote_str(self.format_arg(res_str).zfill(hex_digits))
def result(self) -> List[str]:
result = self.int_a * self.int_b
return [self.format_result(result)]
class BignumCoreMontmul(BignumCoreTarget, test_data_generation.BaseTest):
"""Test cases for Montgomery multiplication."""
count = 0
test_function = "mpi_core_montmul"
test_name = "mbedtls_mpi_core_montmul"
start_2_mpi4 = False
start_2_mpi8 = False
replay_test_cases = [
(2, 1, 1, 1, "19", "1", "1D"), (2, 1, 1, 1, "7", "1", "9"),
(2, 1, 1, 1, "4", "1", "9"),
(
12, 1, 6, 1, (
"3C246D0E059A93A266288A7718419EC741661B474C58C032C5EDAF92709402" +
"B07CC8C7CE0B781C641A1EA8DB2F4343"
), "1", (
"66A198186C18C10B2F5ED9B522752A9830B69916E535C8F047518A889A43A5" +
"94B6BED27A168D31D4A52F88925AA8F5"
)
), (
8, 1, 4, 1,
"1E442976B0E63D64FCCE74B999E470CA9888165CB75BFA1F340E918CE03C6211",
"1", "B3A119602EE213CDE28581ECD892E0F592A338655DCE4CA88054B3D124D0E561"
), (
22, 1, 11, 1, (
"7CF5AC97304E0B63C65413F57249F59994B0FED1D2A8D3D83ED5FA38560FFB" +
"82392870D6D08F87D711917FD7537E13B7E125BE407E74157776839B0AC9DB" +
"23CBDFC696104353E4D2780B2B4968F8D8542306BCA7A2366E"
), "1", (
"284139EA19C139EBE09A8111926AAA39A2C2BE12ED487A809D3CB5BC558547" +
"25B4CDCB5734C58F90B2F60D99CC1950CDBC8D651793E93C9C6F0EAD752500" +
"A32C56C62082912B66132B2A6AA42ADA923E1AD22CEB7BA0123"
)
)
] # type: List[Tuple[int, int, int, int, str, str, str]]
random_test_cases = [
("2", "2", "3", ""), ("1", "2", "3", ""), ("2", "1", "3", ""),
("6", "5", "7", ""), ("3", "4", "7", ""), ("1", "6", "7", ""), ("5", "6", "7", ""),
("3", "4", "B", ""), ("7", "4", "B", ""), ("9", "7", "B", ""), ("2", "a", "B", ""),
("25", "16", "29", "(0x29 is prime)"), ("8", "28", "29", ""),
("18", "21", "29", ""), ("15", "f", "29", ""),
("e2", "ea", "FF", ""), ("43", "72", "FF", ""),
("d8", "70", "FF", ""), ("3c", "7c", "FF", ""),
("99", "b9", "101", "(0x101 is prime)"), ("65", "b2", "101", ""),
("81", "32", "101", ""), ("51", "dd", "101", ""),
("d5", "143", "38B", "(0x38B is prime)"), ("3d", "387", "38B", ""),
("160", "2e5", "38B", ""), ("10f", "137", "38B", ""),
("7dac", "25a", "8003", "(0x8003 is prime)"), ("6f1c", "3286", "8003", ""),
("59ed", "2f3f", "8003", ""), ("6893", "736d", "8003", ""),
("d199", "2832", "10001", "(0x10001 is prime)"), ("c3b2", "3e5b", "10001", ""),
("abe4", "214e", "10001", ""), ("4360", "a05d", "10001", ""),
("3f5a1", "165b2", "7F7F7", ""), ("3bd29", "37863", "7F7F7", ""),
("60c47", "64819", "7F7F7", ""), ("16584", "12c49", "7F7F7", ""),
("1ff03f", "610347", "800009", "(0x800009 is prime)"), ("340fd5", "19812e", "800009", ""),
("3fe2e8", "4d0dc7", "800009", ""), ("40356", "e6392", "800009", ""),
("dd8a1d", "266c0e", "100002B", "(0x100002B is prime)"),
("3fa1cb", "847fd6", "100002B", ""), ("5f439d", "5c3196", "100002B", ""),
("18d645", "f72dc6", "100002B", ""),
("20051ad", "37def6e", "37EEE9D", "(0x37EEE9D is prime)"),
("2ec140b", "3580dbf", "37EEE9D", ""), ("1d91b46", "190d4fc", "37EEE9D", ""),
("34e488d", "1224d24", "37EEE9D", ""),
("2a4fe2cb", "263466a9", "8000000B", "(0x8000000B is prime)"),
("5643fe94", "29a1aefa", "8000000B", ""), ("29633513", "7b007ac4", "8000000B", ""),
("2439cef5", "5c9d5a47", "8000000B", ""),
("4de3cfaa", "50dea178", "8CD626B9", "(0x8CD626B9 is prime)"),
("b8b8563", "10dbbbac", "8CD626B9", ""), ("4e8a6151", "5574ec19", "8CD626B9", ""),
("69224878", "309cfc23", "8CD626B9", ""),
("fb6f7fb6", "afb05423", "10000000F", "(0x10000000F is prime)"),
("8391a243", "26034dcd", "10000000F", ""), ("d26b98c", "14b2d6aa", "10000000F", ""),
("6b9f1371", "a21daf1d", "10000000F", ""),
(
"9f49435ad", "c8264ade8", "174876E7E9",
"0x174876E7E9 is prime (dec) 99999999977"
),
("c402da434", "1fb427acf", "174876E7E9", ""),
("f6ebc2bb1", "1096d39f2a", "174876E7E9", ""),
("153b7f7b6b", "878fda8ff", "174876E7E9", ""),
("2c1adbb8d6", "4384d2d3c6", "8000000017", "(0x8000000017 is prime)"),
("2e4f9cf5fb", "794f3443d9", "8000000017", ""),
("149e495582", "3802b8f7b7", "8000000017", ""),
("7b9d49df82", "69c68a442a", "8000000017", ""),
("683a134600", "6dd80ea9f6", "864CB9076D", "(0x864CB9076D is prime)"),
("13a870ff0d", "59b099694a", "864CB9076D", ""),
("37d06b0e63", "4d2147e46f", "864CB9076D", ""),
("661714f8f4", "22e55df507", "864CB9076D", ""),
("2f0a96363", "52693307b4", "F7F7F7F7F7", ""),
("3c85078e64", "f2275ecb6d", "F7F7F7F7F7", ""),
("352dae68d1", "707775b4c6", "F7F7F7F7F7", ""),
("37ae0f3e0b", "912113040f", "F7F7F7F7F7", ""),
("6dada15e31", "f58ed9eff7", "1000000000F", "(0x1000000000F is prime)"),
("69627a7c89", "cfb5ebd13d", "1000000000F", ""),
("a5e1ad239b", "afc030c731", "1000000000F", ""),
("f1cc45f4c5", "c64ad607c8", "1000000000F", ""),
("2ebad87d2e31", "4c72d90bca78", "800000000005", "(0x800000000005 is prime)"),
("a30b3cc50d", "29ac4fe59490", "800000000005", ""),
("33674e9647b4", "5ec7ee7e72d3", "800000000005", ""),
("3d956f474f61", "74070040257d", "800000000005", ""),
("48348e3717d6", "43fcb4399571", "800795D9BA47", "(0x800795D9BA47 is prime)"),
("5234c03cc99b", "2f3cccb87803", "800795D9BA47", ""),
("3ed13db194ab", "44b8f4ba7030", "800795D9BA47", ""),
("1c11e843bfdb", "95bd1b47b08", "800795D9BA47", ""),
("a81d11cb81fd", "1e5753a3f33d", "1000000000015", "(0x1000000000015 is prime)"),
("688c4db99232", "36fc0cf7ed", "1000000000015", ""),
("f0720cc07e07", "fc76140ed903", "1000000000015", ""),
("2ec61f8d17d1", "d270c85e36d2", "1000000000015", ""),
(
"6a24cd3ab63820", "ed4aad55e5e348", "100000000000051",
"(0x100000000000051 is prime)"
),
("e680c160d3b248", "31e0d8840ed510", "100000000000051", ""),
("a80637e9aebc38", "bb81decc4e1738", "100000000000051", ""),
("9afa5a59e9d630", "be9e65a6d42938", "100000000000051", ""),
("ab5e104eeb71c000", "2cffbd639e9fea00", "ABCDEF0123456789", ""),
("197b867547f68a00", "44b796cf94654800", "ABCDEF0123456789", ""),
("329f9483a04f2c00", "9892f76961d0f000", "ABCDEF0123456789", ""),
("4a2e12dfb4545000", "1aa3e89a69794500", "ABCDEF0123456789", ""),
(
"8b9acdf013d140f000", "12e4ceaefabdf2b2f00", "25A55A46E5DA99C71C7",
"0x25A55A46E5DA99C71C7 is the 3rd repunit prime(dec) 11111111111111111111111"
),
("1b8d960ea277e3f5500", "14418aa980e37dd000", "25A55A46E5DA99C71C7", ""),
("7314524977e8075980", "8172fa45618ccd0d80", "25A55A46E5DA99C71C7", ""),
("ca14f031769be63580", "147a2f3cf2964ca9400", "25A55A46E5DA99C71C7", ""),
(
"18532ba119d5cd0cf39735c0000", "25f9838e31634844924733000000",
"314DC643FB763F2B8C0E2DE00879",
"0x314DC643FB763F2B8C0E2DE00879 is (dec)99999999977^3"
),
(
"a56e2d2517519e3970e70c40000", "ec27428d4bb380458588fa80000",
"314DC643FB763F2B8C0E2DE00879", ""
),
(
"1cb5e8257710e8653fff33a00000", "15fdd42fe440fd3a1d121380000",
"314DC643FB763F2B8C0E2DE00879", ""
),
(
"e50d07a65fc6f93e538ce040000", "1f4b059ca609f3ce597f61240000",
"314DC643FB763F2B8C0E2DE00879", ""
),
(
"1ea3ade786a095d978d387f30df9f20000000",
"127c448575f04af5a367a7be06c7da0000000",
"47BF19662275FA2F6845C74942ED1D852E521",
"0x47BF19662275FA2F6845C74942ED1D852E521 is (dec) 99999999977^4"
),
(
"16e15b0ca82764e72e38357b1f10a20000000",
"43e2355d8514bbe22b0838fdc3983a0000000",
"47BF19662275FA2F6845C74942ED1D852E521", ""
),
(
"be39332529d93f25c3d116c004c620000000",
"5cccec42370a0a2c89c6772da801a0000000",
"47BF19662275FA2F6845C74942ED1D852E521", ""
),
(
"ecaa468d90de0eeda474d39b3e1fc0000000",
"1e714554018de6dc0fe576bfd3b5660000000",
"47BF19662275FA2F6845C74942ED1D852E521", ""
),
(
"32298816711c5dce46f9ba06e775c4bedfc770e6700000000000000",
"8ee751fd5fb24f0b4a653cb3a0c8b7d9e724574d168000000000000",
"97EDD86E4B5C4592C6D32064AC55C888A7245F07CA3CC455E07C931",
(
"0x97EDD86E4B5C4592C6D32064AC55C888A7245F07CA3CC455E07C931" +
" is (dec) 99999999977^6"
)
),
(
"29213b9df3cfd15f4b428645b67b677c29d1378d810000000000000",
"6cbb732c65e10a28872394dfdd1936d5171c3c3aac0000000000000",
"97EDD86E4B5C4592C6D32064AC55C888A7245F07CA3CC455E07C931", ""
),
(
"6f18db06ad4abc52c0c50643dd13098abccd4a232f0000000000000",
"7e6bf41f2a86098ad51f98dfc10490ba3e8081bc830000000000000",
"97EDD86E4B5C4592C6D32064AC55C888A7245F07CA3CC455E07C931", ""
),
(
"62d3286cd706ad9d73caff63f1722775d7e8c731208000000000000",
"530f7ba02ae2b04c2fe3e3d27ec095925631a6c2528000000000000",
"97EDD86E4B5C4592C6D32064AC55C888A7245F07CA3CC455E07C931", ""
),
(
"a6c6503e3c031fdbf6009a89ed60582b7233c5a85de28b16000000000000000",
"75c8ed18270b583f16d442a467d32bf95c5e491e9b8523798000000000000000",
"DD15FE80B731872AC104DB37832F7E75A244AA2631BC87885B861E8F20375499",
(
"0xDD15FE80B731872AC104DB37832F7E75A244AA2631BC87885B861E8F20375499" +
" is (dec) 99999999977^7"
)
),
(
"bf84d1f85cf6b51e04d2c8f4ffd03532d852053cf99b387d4000000000000000",
"397ba5a743c349f4f28bc583ecd5f06e0a25f9c6d98f09134000000000000000",
"DD15FE80B731872AC104DB37832F7E75A244AA2631BC87885B861E8F20375499", ""
),
(
"6db11c3a4152ed1a2aa6fa34b0903ec82ea1b88908dcb482000000000000000",
"ac8ac576a74ad6ca48f201bf89f77350ce86e821358d85920000000000000000",
"DD15FE80B731872AC104DB37832F7E75A244AA2631BC87885B861E8F20375499", ""
),
(
"3001d96d7fe8b733f33687646fc3017e3ac417eb32e0ec708000000000000000",
"925ddbdac4174e8321a48a32f79640e8cf7ec6f46ea235a80000000000000000",
"DD15FE80B731872AC104DB37832F7E75A244AA2631BC87885B861E8F20375499", ""
),
(
"1029048755f2e60dd98c8de6d9989226b6bb4f0db8e46bd1939de560000000000000000000",
"51bb7270b2e25cec0301a03e8275213bb6c2f6e6ec93d4d46d36ca0000000000000000000",
"141B8EBD9009F84C241879A1F680FACCED355DA36C498F73E96E880CF78EA5F96146380E41",
(
"0x141B8EBD9009F84C241879A1F680FACCED355DA36C498F73E96E880CF78EA5F96146" +
"380E41 is 99999999977^8"
)
),
(
"1c5337ff982b3ad6611257dbff5bbd7a9920ba2d4f5838a0cc681ce000000000000000000",
"520c5d049ca4702031ba728591b665c4d4ccd3b2b86864d4c160fd2000000000000000000",
"141B8EBD9009F84C241879A1F680FACCED355DA36C498F73E96E880CF78EA5F96146380E41",
""
),
(
"57074dfa00e42f6555bae624b7f0209f218adf57f73ed34ab0ff90c000000000000000000",
"41eb14b6c07bfd3d1fe4f4a610c17cc44fcfcda695db040e011065000000000000000000",
"141B8EBD9009F84C241879A1F680FACCED355DA36C498F73E96E880CF78EA5F96146380E41",
""
),
(
"d8ed7feed2fe855e6997ad6397f776158573d425031bf085a615784000000000000000000",
"6f121dcd18c578ab5e229881006007bb6d319b179f11015fe958b9c000000000000000000",
"141B8EBD9009F84C241879A1F680FACCED355DA36C498F73E96E880CF78EA5F96146380E41",
""
),
(
(
"2a462b156180ea5fe550d3758c764e06fae54e626b5f503265a09df76edbdfbf" +
"a1e6000000000000000000000000"
), (
"1136f41d1879fd4fb9e49e0943a46b6704d77c068ee237c3121f9071cfd3e6a0" +
"0315800000000000000000000000"
), (
"2A94608DE88B6D5E9F8920F5ABB06B24CC35AE1FBACC87D075C621C3E2833EC90" +
"2713E40F51E3B3C214EDFABC451"
), (
"0x2A94608DE88B6D5E9F8920F5ABB06B24CC35AE1FBACC87D075C621C3E2833EC" +
"902713E40F51E3B3C214EDFABC451 is (dec) 99999999977^10"
)
),
(
(
"c1ac3800dfb3c6954dea391d206200cf3c47f795bf4a5603b4cb88ae7e574de47" +
"40800000000000000000000000"
), (
"c0d16eda0549ede42fa0deb4635f7b7ce061fadea02ee4d85cba4c4f709603419" +
"3c800000000000000000000000"
), (
"2A94608DE88B6D5E9F8920F5ABB06B24CC35AE1FBACC87D075C621C3E2833EC90" +
"2713E40F51E3B3C214EDFABC451"
), ""
),
(
(
"19e45bb7633094d272588ad2e43bcb3ee341991c6731b6fa9d47c4018d7ce7bba" +
"5ee800000000000000000000000"
), (
"1e4f83166ae59f6b9cc8fd3e7677ed8bfc01bb99c98bd3eb084246b64c1e18c33" +
"65b800000000000000000000000"
), (
"2A94608DE88B6D5E9F8920F5ABB06B24CC35AE1FBACC87D075C621C3E2833EC90" +
"2713E40F51E3B3C214EDFABC451"
), ""
),
(
(
"1aa93395fad5f9b7f20b8f9028a054c0bb7c11bb8520e6a95e5a34f06cb70bcdd" +
"01a800000000000000000000000"
), (
"54b45afa5d4310192f8d224634242dd7dcfb342318df3d9bd37b4c614788ba13b" +
"8b000000000000000000000000"
), (
"2A94608DE88B6D5E9F8920F5ABB06B24CC35AE1FBACC87D075C621C3E2833EC90" +
"2713E40F51E3B3C214EDFABC451"
), ""
),
(
(
"544f2628a28cfb5ce0a1b7180ee66b49716f1d9476c466c57f0c4b23089917843" +
"06d48f78686115ee19e25400000000000000000000000000000000"
), (
"677eb31ef8d66c120fa872a60cd47f6e10cbfdf94f90501bd7883cba03d185be0" +
"a0148d1625745e9c4c827300000000000000000000000000000000"
), (
"8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBFA1" +
"1DABD6E6144BEF37C6800000000000000000000000000000000051"
), (
"0x8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBF" +
"A11DABD6E6144BEF37C6800000000000000000000000000000000051 is prime," +
" (dec) 10^143 + 3^4"
)
),
(
(
"76bb3470985174915e9993522aec989666908f9e8cf5cb9f037bf4aee33d8865c" +
"b6464174795d07e30015b80000000000000000000000000000000"
), (
"6aaaf60d5784dcef612d133613b179a317532ecca0eed40b8ad0c01e6d4a6d8c7" +
"9a52af190abd51739009a900000000000000000000000000000000"
), (
"8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBFA1" +
"1DABD6E6144BEF37C6800000000000000000000000000000000051"
), ""
),
(
(
"6cfdd6e60912e441d2d1fc88f421b533f0103a5322ccd3f4db84861643ad63fd6" +
"3d1d8cfbc1d498162786ba00000000000000000000000000000000"
), (
"1177246ec5e93814816465e7f8f248b350d954439d35b2b5d75d917218e7fd5fb" +
"4c2f6d0667f9467fdcf33400000000000000000000000000000000"
), (
"8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBFA1" +
"1DABD6E6144BEF37C6800000000000000000000000000000000051"
), ""
),
(
(
"7a09a0b0f8bbf8057116fb0277a9bdf3a91b5eaa8830d448081510d8973888be5" +
"a9f0ad04facb69aa3715f00000000000000000000000000000000"
), (
"764dec6c05a1c0d87b649efa5fd94c91ea28bffb4725d4ab4b33f1a3e8e3b314d" +
"799020e244a835a145ec9800000000000000000000000000000000"
), (
"8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBFA1" +
"1DABD6E6144BEF37C6800000000000000000000000000000000051"
), ""
)
] # type: List[Tuple[str, str, str, str]]
def __init__(
self, val_a: str, val_b: str, val_n: str, case_description: str = ""
):
self.case_description = case_description
self.arg_a = val_a
self.int_a = bignum_common.hex_to_int(val_a)
self.arg_b = val_b
self.int_b = bignum_common.hex_to_int(val_b)
self.arg_n = val_n
self.int_n = bignum_common.hex_to_int(val_n)
limbs_a4 = bignum_common.limbs_mpi(self.int_a, 32)
limbs_a8 = bignum_common.limbs_mpi(self.int_a, 64)
self.limbs_b4 = bignum_common.limbs_mpi(self.int_b, 32)
self.limbs_b8 = bignum_common.limbs_mpi(self.int_b, 64)
self.limbs_an4 = bignum_common.limbs_mpi(self.int_n, 32)
self.limbs_an8 = bignum_common.limbs_mpi(self.int_n, 64)
if limbs_a4 > self.limbs_an4 or limbs_a8 > self.limbs_an8:
raise Exception("Limbs of input A ({}) exceeds N ({})".format(
self.arg_a, self.arg_n
))
def arguments(self) -> List[str]:
return [
str(self.limbs_an4), str(self.limbs_b4),
str(self.limbs_an8), str(self.limbs_b8),
bignum_common.quote_str(self.arg_a),
bignum_common.quote_str(self.arg_b),
bignum_common.quote_str(self.arg_n)
] + self.result()
def description(self) -> str:
if self.case_description != "replay":
if not self.start_2_mpi4 and self.limbs_an4 > 1:
tmp = "(start of 2-MPI 4-byte bignums) "
self.__class__.start_2_mpi4 = True
elif not self.start_2_mpi8 and self.limbs_an8 > 1:
tmp = "(start of 2-MPI 8-byte bignums) "
self.__class__.start_2_mpi8 = True
else:
tmp = "(gen) "
self.case_description = tmp + self.case_description
return super().description()
def result(self) -> List[str]:
"""Get the result of the operation."""
r4 = bignum_common.bound_mpi_limbs(self.limbs_an4, 32)
i4 = bignum_common.invmod(r4, self.int_n)
x4 = self.int_a * self.int_b * i4
x4 = x4 % self.int_n
r8 = bignum_common.bound_mpi_limbs(self.limbs_an8, 64)
i8 = bignum_common.invmod(r8, self.int_n)
x8 = self.int_a * self.int_b * i8
x8 = x8 % self.int_n
return [
"\"{:x}\"".format(x4),
"\"{:x}\"".format(x8)
]
def set_limbs(
self, limbs_an4: int, limbs_b4: int, limbs_an8: int, limbs_b8: int
) -> None:
"""Set number of limbs for each input.
Replaces default values set during initialization.
"""
self.limbs_an4 = limbs_an4
self.limbs_b4 = limbs_b4
self.limbs_an8 = limbs_an8
self.limbs_b8 = limbs_b8
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
"""Generate replay and randomly generated test cases."""
# Test cases which replay captured invocations during unit test runs.
for limbs_an4, limbs_b4, limbs_an8, limbs_b8, a, b, n in cls.replay_test_cases:
cur_op = cls(a, b, n, case_description="replay")
cur_op.set_limbs(limbs_an4, limbs_b4, limbs_an8, limbs_b8)
yield cur_op.create_test_case()
# Random test cases can be generated using mpi_modmul_case_generate()
# Uses a mixture of primes and odd numbers as N, with four randomly
# generated cases for each N.
for a, b, n, description in cls.random_test_cases:
cur_op = cls(a, b, n, case_description=description)
yield cur_op.create_test_case()
def mpi_modmul_case_generate() -> None:
"""Generate valid inputs for montmul tests using moduli.
For each modulus, generates random values for A and B and simple descriptions
for the test case.
"""
moduli = [
("3", ""), ("7", ""), ("B", ""), ("29", ""), ("FF", ""),
("101", ""), ("38B", ""), ("8003", ""), ("10001", ""),
("7F7F7", ""), ("800009", ""), ("100002B", ""), ("37EEE9D", ""),
("8000000B", ""), ("8CD626B9", ""), ("10000000F", ""),
("174876E7E9", "is prime (dec) 99999999977"),
("8000000017", ""), ("864CB9076D", ""), ("F7F7F7F7F7", ""),
("1000000000F", ""), ("800000000005", ""), ("800795D9BA47", ""),
("1000000000015", ""), ("100000000000051", ""), ("ABCDEF0123456789", ""),
(
"25A55A46E5DA99C71C7",
"is the 3rd repunit prime (dec) 11111111111111111111111"
),
("314DC643FB763F2B8C0E2DE00879", "is (dec)99999999977^3"),
("47BF19662275FA2F6845C74942ED1D852E521", "is (dec) 99999999977^4"),
(
"97EDD86E4B5C4592C6D32064AC55C888A7245F07CA3CC455E07C931",
"is (dec) 99999999977^6"
),
(
"DD15FE80B731872AC104DB37832F7E75A244AA2631BC87885B861E8F20375499",
"is (dec) 99999999977^7"
),
(
"141B8EBD9009F84C241879A1F680FACCED355DA36C498F73E96E880CF78EA5F96146380E41",
"is (dec) 99999999977^8"
),
(
(
"2A94608DE88B6D5E9F8920F5ABB06B24CC35AE1FBACC87D075C621C3E283" +
"3EC902713E40F51E3B3C214EDFABC451"
),
"is (dec) 99999999977^10"
),
(
"8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBFA11" +
"DABD6E6144BEF37C6800000000000000000000000000000000051",
"is prime, (dec) 10^143 + 3^4"
)
] # type: List[Tuple[str, str]]
primes = [
"3", "7", "B", "29", "101", "38B", "8003", "10001", "800009",
"100002B", "37EEE9D", "8000000B", "8CD626B9",
# From here they require > 1 4-byte MPI
"10000000F", "174876E7E9", "8000000017", "864CB9076D", "1000000000F",
"800000000005", "800795D9BA47", "1000000000015", "100000000000051",
# From here they require > 1 8-byte MPI
"25A55A46E5DA99C71C7", # this is 11111111111111111111111 decimal
# 10^143 + 3^4: (which is prime)
# 100000000000000000000000000000000000000000000000000000000000000000000000000000
# 000000000000000000000000000000000000000000000000000000000000000081
(
"8335616AED761F1F7F44E6BD49E807B82E3BF2BF11BFA6AF813C808DBF33DBFA11" +
"DABD6E6144BEF37C6800000000000000000000000000000000051"
)
] # type: List[str]
generated_inputs = []
for mod, description in moduli:
n = bignum_common.hex_to_int(mod)
mod_read = "{:x}".format(n)
case_count = 3 if n < 5 else 4
cases = {} # type: Dict[int, int]
i = 0
while i < case_count:
a = random.randint(1, n)
b = random.randint(1, n)
if cases.get(a) == b:
continue
cases[a] = b
if description:
out_description = "0x{} {}".format(mod_read, description)
elif i == 0 and len(mod) > 1 and mod in primes:
out_description = "(0x{} is prime)"
else:
out_description = ""
generated_inputs.append(
("{:x}".format(a), "{:x}".format(b), mod, out_description)
)
i += 1
print(generated_inputs)
class BignumCoreExpMod(BignumCoreTarget, bignum_common.ModOperationCommon):
"""Test cases for bignum core exponentiation."""
symbol = "^"
test_function = "mpi_core_exp_mod"
test_name = "Core modular exponentiation (Mongtomery form only)"
input_style = "fixed"
montgomery_form_a = True
def result(self) -> List[str]:
# Result has to be given in Montgomery form too
result = pow(self.int_a, self.int_b, self.int_n)
mont_result = self.to_montgomery(result)
return [self.format_result(mont_result)]
@property
def is_valid(self) -> bool:
# The base needs to be canonical, but the exponent can be larger than
# the modulus (see for example exponent blinding)
return bool(self.int_a < self.int_n)
class BignumCoreSubInt(BignumCoreTarget, bignum_common.OperationCommon):
"""Test cases for bignum core sub int."""
count = 0
symbol = "-"
test_function = "mpi_core_sub_int"
test_name = "mpi_core_sub_int"
input_style = "arch_split"
@property
def is_valid(self) -> bool:
# This is "sub int", so b is only one limb
if bignum_common.limbs_mpi(self.int_b, self.bits_in_limb) > 1:
return False
return True
# Overriding because we don't want leading zeros on b
@property
def arg_b(self) -> str:
return self.val_b
def result(self) -> List[str]:
result = self.int_a - self.int_b
borrow, result = divmod(result, self.limb_boundary)
# Borrow will be -1 if non-zero, but we want it to be 1 in the test data
return [
self.format_result(result),
str(-borrow)
]
class BignumCoreZeroCheckCT(BignumCoreTarget, bignum_common.OperationCommon):
"""Test cases for bignum core zero check (constant flow)."""
count = 0
symbol = "== 0"
test_function = "mpi_core_check_zero_ct"
test_name = "mpi_core_check_zero_ct"
input_style = "variable"
arity = 1
suffix = True
def result(self) -> List[str]:
result = 1 if self.int_a == 0 else 0
return [str(result)]
class BignumCoreGcdModinvOdd(BignumCoreTarget, test_data_generation.BaseTest):
"""Test cases for bignum core GCD+modinv (odd modulus)"""
test_function = "mpi_core_gcd_modinv_odd"
test_name = "mpi_core_gcd_modinv_odd"
# - All small integers because that naturally covers a lot of cases.
# - (Close to) powers of 2 because that looks like interesting values.
# Also covers the cases where N, N+1 or N-1 is a multiple of A (with
# multiple limbs).
# - X * 2, X * 3 is so that we get GCD(X*2, X*3) = X where the GCD has a
# the same order of magnitude as the inputs.
# - Random values of cryptographic size for good measure.
DATA = (
("0", 0),
("1", 1),
("2", 2),
("3", 3),
("4", 4),
("5", 5),
("6", 6),
("7", 7),
("2^64 - 1", 2**64 - 1),
("2^64", 2**64),
("2^64 + 1", 2**64 + 1),
("2^128 - 1", 2**128 - 1),
("2^128", 2**128),
("2^128 + 1", 2**128 + 1),
("prime192[1]", int(bignum_data.SAFE_PRIME_192_BIT_SEED_1, 16)),
("prime192[1] * 2", int(bignum_data.SAFE_PRIME_192_BIT_SEED_1, 16) * 2),
("prime192[1] * 3", int(bignum_data.SAFE_PRIME_192_BIT_SEED_1, 16) * 3),
("rand192[2.1]", int(bignum_data.RANDOM_192_BIT_SEED_2_NO1, 16)),
("rand192[2.2]", int(bignum_data.RANDOM_192_BIT_SEED_2_NO2, 16)),
("rand192[2.3]", int(bignum_data.RANDOM_192_BIT_SEED_2_NO3, 16)),
("rand192[2.4]", int(bignum_data.RANDOM_192_BIT_SEED_2_NO4, 16)),
("rand192[2.9]", int(bignum_data.RANDOM_192_BIT_SEED_2_NO9, 16)),
("prime1024[3]", int(bignum_data.SAFE_PRIME_1024_BIT_SEED_3, 16)),
("rand1024[4.1]", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO1, 16)),
("rand1024[4.2]", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO2, 16)),
("rand1024[4.3]", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO3, 16)),
("rand1024[4.4]", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO4, 16)),
("rand1024[4.5]", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO5, 16)),
("rand1024[4.5] * 2", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO5, 16) * 2),
("rand1024[4.5] * 3", int(bignum_data.RANDOM_1024_BIT_SEED_4_NO5, 16) * 3),
)
def __init__(self, a: int, a_desc: str, n: int, n_desc: str) -> None:
self.a_val = a
self.a_desc = a_desc
self.n_val = n
self.n_desc = n_desc
self.g_val = math.gcd(a, n)
test_i = self.g_val == 1 and self.n_val != 1
self.i_val = bignum_common.invmod_positive(a, n) if test_i else None
def arguments(self) -> List[str]:
a_str = f"{self.a_val:x}"
n_str = f"{self.n_val:x}"
g_str = f"{self.g_val:x}"
i_str = f"{self.i_val:x}" if self.i_val is not None else ""
return [bignum_common.quote_str(s) for s in (a_str, n_str, g_str, i_str)]
def description(self) -> str:
return f"GCD-modinv, A = {self.a_desc}, N = {self.n_desc}"
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
for n_desc, n in cls.DATA:
if n % 2 == 0:
continue
for a_desc, a in cls.DATA:
if a > n:
continue
yield cls(a, a_desc, n, n_desc).create_test_case()

View File

@@ -0,0 +1,159 @@
"""Base values and datasets for bignum generated tests and helper functions that
produced them."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import random
# Functions calling these were used to produce test data and are here only for
# reproducibility, they are not used by the test generation framework/classes
try:
from Cryptodome.Util.number import isPrime, getPrime #type: ignore #pylint: disable=import-error
except ImportError:
pass
# Generated by bignum_common.gen_safe_prime(192,1)
SAFE_PRIME_192_BIT_SEED_1 = "d1c127a667786703830500038ebaef20e5a3e2dc378fb75b"
# First number generated by random.getrandbits(192) - seed(2,2), not a prime
RANDOM_192_BIT_SEED_2_NO1 = "177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"
# Second number generated by random.getrandbits(192) - seed(2,2), not a prime
RANDOM_192_BIT_SEED_2_NO2 = "cf1822ffbc6887782b491044d5e341245c6e433715ba2bdd"
# Third number generated by random.getrandbits(192) - seed(2,2), not a prime
RANDOM_192_BIT_SEED_2_NO3 = "3653f8dd9b1f282e4067c3584ee207f8da94e3e8ab73738f"
# Fourth number generated by random.getrandbits(192) - seed(2,2), not a prime
RANDOM_192_BIT_SEED_2_NO4 = "ffed9235288bc781ae66267594c9c9500925e4749b575bd1"
# Ninth number generated by random.getrandbits(192) - seed(2,2), not a prime
RANDOM_192_BIT_SEED_2_NO9 = "2a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"
# Generated by bignum_common.gen_safe_prime(1024,3)
SAFE_PRIME_1024_BIT_SEED_3 = ("c93ba7ec74d96f411ba008bdb78e63ff11bb5df46a51e16b"
"2c9d156f8e4e18abf5e052cb01f47d0d1925a77f60991577"
"e128fb6f52f34a27950a594baadd3d8057abeb222cf3cca9"
"62db16abf79f2ada5bd29ab2f51244bf295eff9f6aaba130"
"2efc449b128be75eeaca04bc3c1a155d11d14e8be32a2c82"
"87b3996cf6ad5223")
# First number generated by random.getrandbits(1024) - seed(4,2), not a prime
RANDOM_1024_BIT_SEED_4_NO1 = ("6905269ed6f0b09f165c8ce36e2f24b43000de01b2ed40ed"
"3addccb2c33be0ac79d679346d4ac7a5c3902b38963dc6e8"
"534f45738d048ec0f1099c6c3e1b258fd724452ccea71ff4"
"a14876aeaff1a098ca5996666ceab360512bd13110722311"
"710cf5327ac435a7a97c643656412a9b8a1abcd1a6916c74"
"da4f9fc3c6da5d7")
# Second number generated by random.getrandbits(1024) - seed(4,2), not a prime
RANDOM_1024_BIT_SEED_4_NO2 = ("f1cfd99216df648647adec26793d0e453f5082492d83a823"
"3fb62d2c81862fc9634f806fabf4a07c566002249b191bf4"
"d8441b5616332aca5f552773e14b0190d93936e1daca3c06"
"f5ff0c03bb5d7385de08caa1a08179104a25e4664f5253a0"
"2a3187853184ff27459142deccea264542a00403ce80c4b0"
"a4042bb3d4341aad")
# Third number generated by random.getrandbits(1024) - seed(4,2), not a prime
RANDOM_1024_BIT_SEED_4_NO3 = ("14c15c910b11ad28cc21ce88d0060cc54278c2614e1bcb38"
"3bb4a570294c4ea3738d243a6e58d5ca49c7b59b995253fd"
"6c79a3de69f85e3131f3b9238224b122c3e4a892d9196ada"
"4fcfa583e1df8af9b474c7e89286a1754abcb06ae8abb93f"
"01d89a024cdce7a6d7288ff68c320f89f1347e0cdd905ecf"
"d160c5d0ef412ed6")
# Fourth number generated by random.getrandbits(1024) - seed(4,2), not a prime
RANDOM_1024_BIT_SEED_4_NO4 = ("32decd6b8efbc170a26a25c852175b7a96b98b5fbf37a2be"
"6f98bca35b17b9662f0733c846bbe9e870ef55b1a1f65507"
"a2909cb633e238b4e9dd38b869ace91311021c9e32111ac1"
"ac7cc4a4ff4dab102522d53857c49391b36cc9aa78a330a1"
"a5e333cb88dcf94384d4cd1f47ca7883ff5a52f1a05885ac"
"7671863c0bdbc23a")
# Fifth number generated by random.getrandbits(1024) - seed(4,2), not a prime
RANDOM_1024_BIT_SEED_4_NO5 = ("53be4721f5b9e1f5acdac615bc20f6264922b9ccf469aef8"
"f6e7d078e55b85dd1525f363b281b8885b69dc230af5ac87"
"0692b534758240df4a7a03052d733dcdef40af2e54c0ce68"
"1f44ebd13cc75f3edcb285f89d8cf4d4950b16ffc3e1ac3b"
"4708d9893a973000b54a23020fc5b043d6e4a51519d9c9cc"
"52d32377e78131c1")
# Adding 192 bit and 1024 bit numbers because these are the shortest required
# for ECC and RSA respectively.
INPUTS_DEFAULT = [
"0", "1", # corner cases
"2", "3", # small primes
"4", # non-prime even
"38", # small random
SAFE_PRIME_192_BIT_SEED_1, # prime
RANDOM_192_BIT_SEED_2_NO1, # not a prime
RANDOM_192_BIT_SEED_2_NO2, # not a prime
SAFE_PRIME_1024_BIT_SEED_3, # prime
RANDOM_1024_BIT_SEED_4_NO1, # not a prime
RANDOM_1024_BIT_SEED_4_NO3, # not a prime
RANDOM_1024_BIT_SEED_4_NO2, # largest (not a prime)
]
ADD_SUB_DATA = [
"0", "1", "3", "f", "fe", "ff", "100", "ff00",
"fffe", "ffff", "10000", # 2^16 - 1, 2^16, 2^16 + 1
"fffffffe", "ffffffff", "100000000", # 2^32 - 1, 2^32, 2^32 + 1
"1f7f7f7f7f7f7f",
"8000000000000000", "fefefefefefefefe",
"fffffffffffffffe", "ffffffffffffffff", "10000000000000000", # 2^64 - 1, 2^64, 2^64 + 1
"1234567890abcdef0",
"fffffffffffffffffffffffe",
"ffffffffffffffffffffffff",
"1000000000000000000000000",
"fffffffffffffffffefefefefefefefe",
"fffffffffffffffffffffffffffffffe",
"ffffffffffffffffffffffffffffffff",
"100000000000000000000000000000000",
"1234567890abcdef01234567890abcdef0",
"fffffffffffffffffffffffffffffffffffffffffffffffffefefefefefefefe",
"fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe",
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
"10000000000000000000000000000000000000000000000000000000000000000",
"1234567890abcdef01234567890abcdef01234567890abcdef01234567890abcdef0",
]
# Only odd moduli are present as in the new bignum code only odd moduli are
# supported for now.
MODULI_DEFAULT = [
"53", # safe prime
"45", # non-prime
SAFE_PRIME_192_BIT_SEED_1, # safe prime
RANDOM_192_BIT_SEED_2_NO4, # not a prime
SAFE_PRIME_1024_BIT_SEED_3, # safe prime
RANDOM_1024_BIT_SEED_4_NO5, # not a prime
]
# Some functions, e.g. mbedtls_mpi_mod_raw_inv_prime(), only support prime moduli.
ONLY_PRIME_MODULI = [
"53", # safe prime
"8ac72304057392b5", # 9999999997777777333 (longer, not safe, prime)
# The next prime has a different R in Montgomery form depending on
# whether 32- or 64-bit MPIs are used.
"152d02c7e14af67fe0bf", # 99999999999999999991999
SAFE_PRIME_192_BIT_SEED_1, # safe prime
SAFE_PRIME_1024_BIT_SEED_3, # safe prime
]
def __gen_safe_prime(bits, seed):
'''
Generate a safe prime.
This function is intended for generating constants offline and shouldn't be
used in test generation classes.
Requires pycryptodomex for getPrime and isPrime and python 3.9 or later for
randbytes.
'''
rng = random.Random()
# We want reproducibility across python versions
rng.seed(seed, version=2)
while True:
prime = 2*getPrime(bits-1, rng.randbytes)+1 #pylint: disable=no-member
if isPrime(prime, 1e-30):
return prime

View File

@@ -0,0 +1,102 @@
"""Framework classes for generation of bignum mod test cases."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
from typing import List
from . import test_data_generation
from . import bignum_common
from .bignum_data import ONLY_PRIME_MODULI
class BignumModTarget(test_data_generation.BaseTarget):
#pylint: disable=abstract-method, too-few-public-methods
"""Target for bignum mod test case generation."""
target_basename = 'test_suite_bignum_mod.generated'
class BignumModMul(bignum_common.ModOperationCommon,
BignumModTarget):
# pylint:disable=duplicate-code
"""Test cases for bignum mpi_mod_mul()."""
symbol = "*"
test_function = "mpi_mod_mul"
test_name = "mbedtls_mpi_mod_mul"
input_style = "arch_split"
arity = 2
def arguments(self) -> List[str]:
return [self.format_result(self.to_montgomery(self.int_a)),
self.format_result(self.to_montgomery(self.int_b)),
bignum_common.quote_str(self.arg_n)
] + self.result()
def result(self) -> List[str]:
result = (self.int_a * self.int_b) % self.int_n
return [self.format_result(self.to_montgomery(result))]
class BignumModSub(bignum_common.ModOperationCommon, BignumModTarget):
"""Test cases for bignum mpi_mod_sub()."""
symbol = "-"
test_function = "mpi_mod_sub"
test_name = "mbedtls_mpi_mod_sub"
input_style = "fixed"
arity = 2
def result(self) -> List[str]:
result = (self.int_a - self.int_b) % self.int_n
# To make negative tests easier, append 0 for success to the
# generated cases
return [self.format_result(result), "0"]
class BignumModInvNonMont(bignum_common.ModOperationCommon, BignumModTarget):
"""Test cases for bignum mpi_mod_inv() - not in Montgomery form."""
moduli = ONLY_PRIME_MODULI # for now only prime moduli supported
symbol = "^ -1"
test_function = "mpi_mod_inv_non_mont"
test_name = "mbedtls_mpi_mod_inv non-Mont. form"
input_style = "fixed"
arity = 1
suffix = True
disallow_zero_a = True
def result(self) -> List[str]:
result = bignum_common.invmod_positive(self.int_a, self.int_n)
# To make negative tests easier, append 0 for success to the
# generated cases
return [self.format_result(result), "0"]
class BignumModInvMont(bignum_common.ModOperationCommon, BignumModTarget):
"""Test cases for bignum mpi_mod_inv() - Montgomery form."""
moduli = ONLY_PRIME_MODULI # for now only prime moduli supported
symbol = "^ -1"
test_function = "mpi_mod_inv_mont"
test_name = "mbedtls_mpi_mod_inv Mont. form"
input_style = "arch_split" # Mont. form requires arch_split
arity = 1
suffix = True
disallow_zero_a = True
montgomery_form_a = True
def result(self) -> List[str]:
result = bignum_common.invmod_positive(self.int_a, self.int_n)
mont_result = self.to_montgomery(result)
# To make negative tests easier, append 0 for success to the
# generated cases
return [self.format_result(mont_result), "0"]
class BignumModAdd(bignum_common.ModOperationCommon, BignumModTarget):
"""Test cases for bignum mpi_mod_add()."""
count = 0
symbol = "+"
test_function = "mpi_mod_add"
test_name = "mbedtls_mpi_mod_add"
input_style = "fixed"
def result(self) -> List[str]:
result = (self.int_a + self.int_b) % self.int_n
# To make negative tests easier, append "0" for success to the
# generated cases
return [self.format_result(result), "0"]

View File

@@ -0,0 +1,242 @@
"""Framework classes for generation of bignum mod_raw test cases."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
from typing import Iterator, List
from . import test_case
from . import test_data_generation
from . import bignum_common
from .bignum_data import ONLY_PRIME_MODULI
class BignumModRawTarget(test_data_generation.BaseTarget):
#pylint: disable=abstract-method, too-few-public-methods
"""Target for bignum mod_raw test case generation."""
target_basename = 'test_suite_bignum_mod_raw.generated'
class BignumModRawSub(bignum_common.ModOperationCommon,
BignumModRawTarget):
"""Test cases for bignum mpi_mod_raw_sub()."""
symbol = "-"
test_function = "mpi_mod_raw_sub"
test_name = "mbedtls_mpi_mod_raw_sub"
input_style = "fixed"
arity = 2
def arguments(self) -> List[str]:
return [bignum_common.quote_str(n) for n in [self.arg_a,
self.arg_b,
self.arg_n]
] + self.result()
def result(self) -> List[str]:
result = (self.int_a - self.int_b) % self.int_n
return [self.format_result(result)]
class BignumModRawFixQuasiReduction(bignum_common.ModOperationCommon,
BignumModRawTarget):
"""Test cases for ecp quasi_reduction()."""
symbol = "-"
test_function = "mpi_mod_raw_fix_quasi_reduction"
test_name = "fix_quasi_reduction"
input_style = "fixed"
arity = 1
# Extend the default values with n < x < 2n
input_values = bignum_common.ModOperationCommon.input_values + [
"73",
# First number generated by random.getrandbits(1024) - seed(3,2)
"ea7b5bf55eb561a4216363698b529b4a97b750923ceb3ffd",
# First number generated by random.getrandbits(1024) - seed(1,2)
("cd447e35b8b6d8fe442e3d437204e52db2221a58008a05a6c4647159c324c985" +
"9b810e766ec9d28663ca828dd5f4b3b2e4b06ce60741c7a87ce42c8218072e8c" +
"35bf992dc9e9c616612e7696a6cecc1b78e510617311d8a3c2ce6f447ed4d57b" +
"1e2feb89414c343c1027c4d1c386bbc4cd613e30d8f16adf91b7584a2265b1f5")
] # type: List[str]
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return bool(self.int_a < 2 * self.int_n)
class BignumModRawMul(bignum_common.ModOperationCommon,
BignumModRawTarget):
"""Test cases for bignum mpi_mod_raw_mul()."""
symbol = "*"
test_function = "mpi_mod_raw_mul"
test_name = "mbedtls_mpi_mod_raw_mul"
input_style = "arch_split"
arity = 2
def arguments(self) -> List[str]:
return [self.format_result(self.to_montgomery(self.int_a)),
self.format_result(self.to_montgomery(self.int_b)),
bignum_common.quote_str(self.arg_n)
] + self.result()
def result(self) -> List[str]:
result = (self.int_a * self.int_b) % self.int_n
return [self.format_result(self.to_montgomery(result))]
class BignumModRawInvPrime(bignum_common.ModOperationCommon,
BignumModRawTarget):
"""Test cases for bignum mpi_mod_raw_inv_prime()."""
moduli = ONLY_PRIME_MODULI
symbol = "^ -1"
test_function = "mpi_mod_raw_inv_prime"
test_name = "mbedtls_mpi_mod_raw_inv_prime (Montgomery form only)"
input_style = "arch_split"
arity = 1
suffix = True
montgomery_form_a = True
disallow_zero_a = True
def result(self) -> List[str]:
result = bignum_common.invmod_positive(self.int_a, self.int_n)
mont_result = self.to_montgomery(result)
return [self.format_result(mont_result)]
class BignumModRawAdd(bignum_common.ModOperationCommon,
BignumModRawTarget):
"""Test cases for bignum mpi_mod_raw_add()."""
symbol = "+"
test_function = "mpi_mod_raw_add"
test_name = "mbedtls_mpi_mod_raw_add"
input_style = "fixed"
arity = 2
def result(self) -> List[str]:
result = (self.int_a + self.int_b) % self.int_n
return [self.format_result(result)]
class BignumModRawConvertRep(bignum_common.ModOperationCommon,
BignumModRawTarget):
# This is an abstract class, it's ok to have unimplemented methods.
#pylint: disable=abstract-method
"""Test cases for representation conversion."""
symbol = ""
input_style = "arch_split"
arity = 1
rep = bignum_common.ModulusRepresentation.INVALID
def set_representation(self, r: bignum_common.ModulusRepresentation) -> None:
self.rep = r
def arguments(self) -> List[str]:
return ([bignum_common.quote_str(self.arg_n), self.rep.symbol(),
bignum_common.quote_str(self.arg_a)] +
self.result())
def description(self) -> str:
base = super().description()
mod_with_rep = 'mod({})'.format(self.rep.name)
return base.replace('mod', mod_with_rep, 1)
@classmethod
def test_cases_for_values(cls, rep: bignum_common.ModulusRepresentation,
n: str, a: str) -> Iterator[test_case.TestCase]:
"""Emit test cases for the given values (if any).
This may emit no test cases if a isn't valid for the modulus n,
or multiple test cases if rep requires different data depending
on the limb size.
"""
for bil in cls.limb_sizes:
test_object = cls(n, a, bits_in_limb=bil)
test_object.set_representation(rep)
# The class is set to having separate test cases for each limb
# size, because the Montgomery representation requires it.
# But other representations don't require it. So for other
# representations, emit a single test case with no dependency
# on the limb size.
if rep is not bignum_common.ModulusRepresentation.MONTGOMERY:
test_object.dependencies = \
[dep for dep in test_object.dependencies
if not dep.startswith('MBEDTLS_HAVE_INT')]
if test_object.is_valid:
yield test_object.create_test_case()
if rep is not bignum_common.ModulusRepresentation.MONTGOMERY:
# A single test case (emitted, or skipped due to invalidity)
# is enough, since this test case doesn't depend on the
# limb size.
break
# The parent class doesn't support non-bignum parameters. So we override
# test generation, in order to have the representation as a parameter.
@classmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
for rep in bignum_common.ModulusRepresentation.supported_representations():
for n in cls.moduli:
for a in cls.input_values:
yield from cls.test_cases_for_values(rep, n, a)
class BignumModRawCanonicalToModulusRep(BignumModRawConvertRep):
"""Test cases for mpi_mod_raw_canonical_to_modulus_rep."""
test_function = "mpi_mod_raw_canonical_to_modulus_rep"
test_name = "Rep canon->mod"
def result(self) -> List[str]:
return [self.format_result(self.convert_from_canonical(self.int_a, self.rep))]
class BignumModRawModulusToCanonicalRep(BignumModRawConvertRep):
"""Test cases for mpi_mod_raw_modulus_to_canonical_rep."""
test_function = "mpi_mod_raw_modulus_to_canonical_rep"
test_name = "Rep mod->canon"
@property
def arg_a(self) -> str:
return self.format_arg("{:x}".format(self.convert_from_canonical(self.int_a, self.rep)))
def result(self) -> List[str]:
return [self.format_result(self.int_a)]
class BignumModRawConvertToMont(bignum_common.ModOperationCommon,
BignumModRawTarget):
""" Test cases for mpi_mod_raw_to_mont_rep(). """
test_function = "mpi_mod_raw_to_mont_rep"
test_name = "Convert into Mont: "
symbol = "R *"
input_style = "arch_split"
arity = 1
def result(self) -> List[str]:
result = self.to_montgomery(self.int_a)
return [self.format_result(result)]
class BignumModRawConvertFromMont(bignum_common.ModOperationCommon,
BignumModRawTarget):
""" Test cases for mpi_mod_raw_from_mont_rep(). """
test_function = "mpi_mod_raw_from_mont_rep"
test_name = "Convert from Mont: "
symbol = "1/R *"
input_style = "arch_split"
arity = 1
def result(self) -> List[str]:
result = self.from_montgomery(self.int_a)
return [self.format_result(result)]
class BignumModRawModNegate(bignum_common.ModOperationCommon,
BignumModRawTarget):
""" Test cases for mpi_mod_raw_neg(). """
test_function = "mpi_mod_raw_neg"
test_name = "Modular negation: "
symbol = "-"
input_style = "arch_split"
arity = 1
def result(self) -> List[str]:
result = (self.int_n - self.int_a) % self.int_n
return [self.format_result(result)]

View File

@@ -0,0 +1,150 @@
"""Mbed TLS build tree information and manipulation.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import os
import inspect
import re
from typing import Optional
def looks_like_tf_psa_crypto_root(path: str) -> bool:
"""Whether the given directory looks like the root of the PSA Crypto source tree."""
try:
with open(os.path.join(path, 'scripts', 'project_name.txt'), 'r') as f:
return f.read() == "TF-PSA-Crypto\n"
except FileNotFoundError:
return False
def looks_like_mbedtls_root(path: str) -> bool:
"""Whether the given directory looks like the root of the Mbed TLS source tree."""
try:
with open(os.path.join(path, 'scripts', 'project_name.txt'), 'r') as f:
return f.read() == "Mbed TLS\n"
except FileNotFoundError:
return False
def looks_like_root(path: str) -> bool:
return looks_like_tf_psa_crypto_root(path) or looks_like_mbedtls_root(path)
def crypto_core_directory(root: Optional[str] = None, relative: Optional[bool] = False) -> str:
"""
Return the path of the directory containing the PSA crypto core
for either TF-PSA-Crypto or Mbed TLS.
Returns either the full path or relative path depending on the
"relative" boolean argument.
"""
if root is None:
root = guess_project_root()
if looks_like_tf_psa_crypto_root(root):
if relative:
return "core"
return os.path.join(root, "core")
elif looks_like_mbedtls_root(root):
if is_mbedtls_3_6():
path = "library"
else:
path = "tf-psa-crypto/core"
if relative:
return path
return os.path.join(root, path)
else:
raise Exception('Neither Mbed TLS nor TF-PSA-Crypto source tree found')
def crypto_library_filename(root: Optional[str] = None) -> str:
"""Return the crypto library filename for either TF-PSA-Crypto or Mbed TLS."""
if root is None:
root = guess_project_root()
if looks_like_tf_psa_crypto_root(root):
return "tfpsacrypto"
elif looks_like_mbedtls_root(root):
return "mbedcrypto"
else:
raise Exception('Neither Mbed TLS nor TF-PSA-Crypto source tree found')
def check_repo_path():
"""Check that the current working directory is the project root, and throw
an exception if not.
"""
if not all(os.path.isdir(d) for d in ["include", "library", "tests"]):
raise Exception("This script must be run from Mbed TLS root")
def chdir_to_root() -> None:
"""Detect the root of the Mbed TLS or TF-PSA-Crypto source tree and change to it.
The current directory must be up to two levels deep inside an Mbed TLS or
TF-PSA-Crypto source tree.
"""
for d in [os.path.curdir,
os.path.pardir,
os.path.join(os.path.pardir, os.path.pardir)]:
if looks_like_root(d):
os.chdir(d)
return
raise Exception('Mbed TLS or TF-PSA-Crypto source tree not found')
def guess_project_root():
"""Guess project source code directory.
Return the first possible project root directory.
"""
dirs = set({})
for frame in inspect.stack():
path = os.path.dirname(frame.filename)
for d in ['.', os.path.pardir] \
+ [os.path.join(*([os.path.pardir]*i)) for i in range(2, 10)]:
d = os.path.abspath(os.path.join(path, d))
if d in dirs:
continue
dirs.add(d)
if looks_like_root(d):
return d
raise Exception('Neither Mbed TLS nor TF-PSA-Crypto source tree found')
def guess_mbedtls_root(root: Optional[str] = None) -> str:
"""Guess Mbed TLS source code directory.
Return the first possible Mbed TLS root directory.
Raise an exception if we are not in Mbed TLS.
"""
if root is None:
root = guess_project_root()
if looks_like_mbedtls_root(root):
return root
else:
raise Exception('Mbed TLS source tree not found')
def guess_tf_psa_crypto_root(root: Optional[str] = None) -> str:
"""Guess TF-PSA-Crypto source code directory.
Return the first possible TF-PSA-Crypto root directory.
Raise an exception if we are not in TF-PSA-Crypto.
"""
if root is None:
root = guess_project_root()
if looks_like_tf_psa_crypto_root(root):
return root
else:
raise Exception('TF-PSA-Crypto source tree not found')
def framework_root(root: Optional[str] = None) -> str:
"""Return the path to the framework directory for this project."""
if root is None:
root = guess_project_root()
return os.path.join(root, 'framework')
def is_mbedtls_3_6() -> bool:
"""Whether the working tree is an Mbed TLS 3.6 one or not
Return false if we are in TF-PSA-Crypto or in Mbed TLS but with a version
different from 3.6.x.
Raise an exception if we are neither in Mbed TLS nor in TF-PSA-Crypto.
"""
root = guess_project_root()
if not looks_like_mbedtls_root(root):
return False
with open(os.path.join(root, 'include', 'mbedtls', 'build_info.h'), 'r') as f:
return re.search(r"#define MBEDTLS_VERSION_NUMBER.*0x0306", f.read()) is not None

View File

@@ -0,0 +1,176 @@
"""Generate and run C code.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import os
import platform
import subprocess
import sys
import tempfile
class CompileError(Exception):
"""Exception to represent an error during the compilation."""
def __init__(self, message):
"""Save the error massage"""
super().__init__(message)
self.message = message
def remove_file_if_exists(filename):
"""Remove the specified file, ignoring errors."""
if not filename:
return
try:
os.remove(filename)
except OSError:
pass
def create_c_file(file_label):
"""Create a temporary C file.
* ``file_label``: a string that will be included in the file name.
Return ```(c_file, c_name, exe_name)``` where ``c_file`` is a Python
stream open for writing to the file, ``c_name`` is the name of the file
and ``exe_name`` is the name of the executable that will be produced
by compiling the file.
"""
c_fd, c_name = tempfile.mkstemp(prefix='tmp-{}-'.format(file_label),
suffix='.c')
exe_suffix = '.exe' if platform.system() == 'Windows' else ''
exe_name = c_name[:-2] + exe_suffix
remove_file_if_exists(exe_name)
c_file = os.fdopen(c_fd, 'w', encoding='ascii')
return c_file, c_name, exe_name
def generate_c_printf_expressions(c_file, cast_to, printf_format, expressions):
"""Generate C instructions to print the value of ``expressions``.
Write the code with ``c_file``'s ``write`` method.
Each expression is cast to the type ``cast_to`` and printed with the
printf format ``printf_format``.
"""
for expr in expressions:
c_file.write(' printf("{}\\n", ({}) {});\n'
.format(printf_format, cast_to, expr))
def generate_c_file(c_file,
caller, header,
main_generator):
"""Generate a temporary C source file.
* ``c_file`` is an open stream on the C source file.
* ``caller``: an informational string written in a comment at the top
of the file.
* ``header``: extra code to insert before any function in the generated
C file.
* ``main_generator``: a function called with ``c_file`` as its sole argument
to generate the body of the ``main()`` function.
"""
c_file.write('/* Generated by {} */'
.format(caller))
c_file.write('''
#include <stdio.h>
''')
c_file.write(header)
c_file.write('''
int main(void)
{
''')
main_generator(c_file)
c_file.write(''' return 0;
}
''')
def compile_c_file(c_filename, exe_filename, include_dirs):
"""Compile a C source file with the host compiler.
* ``c_filename``: the name of the source file to compile.
* ``exe_filename``: the name for the executable to be created.
* ``include_dirs``: a list of paths to include directories to be passed
with the -I switch.
"""
# Respect $HOSTCC if it is set
cc = os.getenv('HOSTCC', None)
if cc is None:
cc = os.getenv('CC', 'cc')
cmd = [cc]
proc = subprocess.Popen(cmd,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
universal_newlines=True)
cc_is_msvc = 'Microsoft (R) C/C++' in proc.communicate()[1]
cmd += ['-I' + dir for dir in include_dirs]
if cc_is_msvc:
# MSVC has deprecated using -o to specify the output file,
# and produces an object file in the working directory by default.
obj_filename = exe_filename[:-4] + '.obj'
cmd += ['-Fe' + exe_filename, '-Fo' + obj_filename]
else:
cmd += ['-o' + exe_filename]
try:
subprocess.check_output(cmd + [c_filename],
stderr=subprocess.PIPE,
universal_newlines=True)
except subprocess.CalledProcessError as e:
raise CompileError(e.stderr) from e
def get_c_expression_values(
cast_to, printf_format,
expressions,
caller=__name__, file_label='',
header='', include_path=None,
keep_c=False,
): # pylint: disable=too-many-arguments, too-many-locals
"""Generate and run a program to print out numerical values for expressions.
* ``cast_to``: a C type.
* ``printf_format``: a printf format suitable for the type ``cast_to``.
* ``header``: extra code to insert before any function in the generated
C file.
* ``expressions``: a list of C language expressions that have the type
``cast_to``.
* ``include_path``: a list of directories containing header files.
* ``keep_c``: if true, keep the temporary C file (presumably for debugging
purposes).
Use the C compiler specified by the ``CC`` environment variable, defaulting
to ``cc``. If ``CC`` looks like MSVC, use its command line syntax,
otherwise assume the compiler supports Unix traditional ``-I`` and ``-o``.
Return the list of values of the ``expressions``.
"""
if include_path is None:
include_path = []
c_name = None
exe_name = None
obj_name = None
try:
c_file, c_name, exe_name = create_c_file(file_label)
generate_c_file(
c_file, caller, header,
lambda c_file: generate_c_printf_expressions(c_file,
cast_to, printf_format,
expressions)
)
c_file.close()
compile_c_file(c_name, exe_name, include_path)
if keep_c:
sys.stderr.write('List of {} tests kept at {}\n'
.format(caller, c_name))
else:
os.remove(c_name)
output = subprocess.check_output([exe_name])
return output.decode('ascii').strip().split('\n')
finally:
remove_file_if_exists(exe_name)
remove_file_if_exists(obj_name)

View File

@@ -0,0 +1,168 @@
"""Helper functions to parse C code in heavily constrained scenarios.
Currently supported functionality:
* read_function_declarations: read function declarations from a header file.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
### WARNING: the code in this file has not been extensively reviewed yet.
### We do not think it is harmful, but it may be below our normal standards
### for robustness and maintainability.
import re
from typing import Dict, Iterable, Iterator, List, Optional, Tuple
class ArgumentInfo:
"""Information about an argument to an API function."""
#pylint: disable=too-few-public-methods
_KEYWORDS = [
'const', 'register', 'restrict',
'int', 'long', 'short', 'signed', 'unsigned',
]
_DECLARATION_RE = re.compile(
r'(?P<type>\w[\w\s*]*?)\s*' +
r'(?!(?:' + r'|'.join(_KEYWORDS) + r'))(?P<name>\b\w+\b)?' +
r'\s*(?P<suffix>\[[^][]*\])?\Z',
re.A | re.S)
@classmethod
def normalize_type(cls, typ: str) -> str:
"""Normalize whitespace in a type."""
typ = re.sub(r'\s+', r' ', typ)
typ = re.sub(r'\s*\*', r' *', typ)
return typ
def __init__(self, decl: str) -> None:
self.decl = decl.strip()
m = self._DECLARATION_RE.match(self.decl)
if not m:
raise ValueError(self.decl)
self.type = self.normalize_type(m.group('type')) #type: str
self.name = m.group('name') #type: Optional[str]
self.suffix = m.group('suffix') if m.group('suffix') else '' #type: str
def __str__(self) -> str:
return self.decl
class FunctionInfo:
"""Information about an API function."""
#pylint: disable=too-few-public-methods
# Regex matching the declaration of a function that returns void.
VOID_RE = re.compile(r'\s*\bvoid\s*\Z', re.A)
def __init__(self, #pylint: disable=too-many-arguments
filename: str,
line_number: int,
qualifiers: Iterable[str],
return_type: str,
name: str,
arguments: List[str],
doc: str = "") -> None:
self.filename = filename
self.line_number = line_number
self.qualifiers = frozenset(qualifiers)
self.return_type = return_type
self.name = name
self.arguments = [ArgumentInfo(arg) for arg in arguments]
self.doc = doc
def returns_void(self) -> bool:
"""Whether the function returns void."""
return bool(self.VOID_RE.search(self.return_type))
def __str__(self) -> str:
str_args = [str(a) for a in self.arguments]
str_text = "{} {} {}({})".format(" ".join(self.qualifiers),
self.return_type, self.name,
", ".join(str_args)).strip()
str_text = self._c_wrap_(str_text)
return self.doc + "\n" + str_text
@staticmethod
def _c_wrap_(in_str: str, line_len: int = 80) -> str:
"""Auto-idents function declaration args using opening parenthesis."""
if len(in_str) >= line_len:
p_idx = in_str.index("(")
ident = " " * p_idx
padded_comma = ",\n" + ident
in_str = in_str.replace(",", padded_comma)
return in_str
# Match one C comment.
# Note that we match both comment types, so things like // in a /*...*/
# comment are handled correctly.
_C_COMMENT_RE = re.compile(r'//(?:[^\n]|\\\n)*|/\*.*?\*/', re.S)
_NOT_NEWLINES_RE = re.compile(r'[^\n]+')
def read_logical_lines(filename: str) -> Iterator[Tuple[int, str]]:
"""Read logical lines from a file.
Logical lines are one or more physical line, with balanced parentheses.
"""
with open(filename, encoding='utf-8') as inp:
content = inp.read()
# Strip comments, but keep newlines for line numbering
content = re.sub(_C_COMMENT_RE,
lambda m: re.sub(_NOT_NEWLINES_RE, "", m.group(0)),
content)
lines = enumerate(content.splitlines(), 1)
for line_number, line in lines:
# Read a logical line, containing balanced parentheses.
# We assume that parentheses are balanced (this should be ok
# since comments have been stripped), otherwise there will be
# a gigantic logical line at the end.
paren_level = line.count('(') - line.count(')')
while paren_level > 0:
_, more = next(lines) #pylint: disable=stop-iteration-return
paren_level += more.count('(') - more.count(')')
line += '\n' + more
yield line_number, line
_C_FUNCTION_DECLARATION_RE = re.compile(
r'(?P<qualifiers>(?:(?:extern|inline|static)\b\s*)*)'
r'(?P<return_type>\w[\w\s*]*?)\s*' +
r'\b(?P<name>\w+)' +
r'\s*\((?P<arguments>.*)\)\s*;',
re.A | re.S)
def read_function_declarations(functions: Dict[str, FunctionInfo],
filename: str) -> None:
"""Collect function declarations from a C header file."""
for line_number, line in read_logical_lines(filename):
m = _C_FUNCTION_DECLARATION_RE.match(line)
if not m:
continue
qualifiers = m.group('qualifiers').split()
return_type = m.group('return_type')
name = m.group('name')
arguments = m.group('arguments').split(',')
if len(arguments) == 1 and re.match(FunctionInfo.VOID_RE, arguments[0]):
arguments = []
# Note: we replace any existing declaration for the same name.
functions[name] = FunctionInfo(filename, line_number,
qualifiers,
return_type,
name,
arguments)
_C_TYPEDEF_DECLARATION_RE = re.compile(r'typedef (?:struct )?(?P<type>\w+) (?P<name>\w+)')
def read_typedefs(filename: str) -> Dict[str, str]:
""" Extract type definitions in a {typedef aliased name: original type} dictionary.
Multi-line typedef struct are not captured. """
type_decl = {}
for _, line in read_logical_lines(filename):
m = _C_TYPEDEF_DECLARATION_RE.match(line)
if m:
type_decl[m.group("name")] = m.group("type")
return type_decl

View File

@@ -0,0 +1,511 @@
"""Generate C wrapper functions.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
### WARNING: the code in this file has not been extensively reviewed yet.
### We do not think it is harmful, but it may be below our normal standards
### for robustness and maintainability.
import os
import re
import sys
from typing import Dict, NamedTuple, List, Optional, Tuple
from .c_parsing_helper import ArgumentInfo, FunctionInfo
from . import typing_util
def c_declare(prefix: str, name: str, suffix: str) -> str:
"""Format a declaration of name with the given type prefix and suffix."""
if not prefix.endswith('*'):
prefix += ' '
return prefix + name + suffix
WrapperInfo = NamedTuple('WrapperInfo', [
('argument_names', List[str]),
('guard', Optional[str]),
('wrapper_name', str),
])
def strip_indentation(in_str: str, new_lines: int = 1, indent_lv: int = 0) -> str:
"""Return a whitespace stripped str, with configurable whitespace in output.
The method will remove space-character indentation from input string.
It will also remove all new-lines around the text-block as well as
trailing whitespace.
The output indentation can be configured by indent_lv, and will use blocks
of 4 spaces.
At the end of the string a `new_lines` amount of empty lines will be added.
"""
_ret_string = in_str.lstrip('\n').rstrip()
# Count empty spaces in beggining of each line. The smallest non-zero entry
# will be used to clean up input indentation.
indents = [len(n)-1 for n in re.findall(r'(?m)^ +\S', in_str)]
if indents:
_ret_string = re.sub(r'(?m)^ {{{indent}}}'.format(indent=min(indents)),
'', _ret_string)
if indent_lv:
_ret_string = '\n'.join([' ' * indent_lv * 4 + s
for s in _ret_string.splitlines()])
return _ret_string + ('\n' * (new_lines + 1))
class Base:
"""Generate a C source file containing wrapper functions."""
# This class is designed to have many methods potentially overloaded.
# Tell pylint not to complain about methods that have unused arguments:
# child classes are likely to override those methods and need the
# arguments in question.
#pylint: disable=no-self-use,unused-argument
# Prefix prepended to the function's name to form the wrapper name.
_WRAPPER_NAME_PREFIX = ''
# Suffix appended to the function's name to form the wrapper name.
_WRAPPER_NAME_SUFFIX = '_wrap'
_INCLUDES = ['<mbedtls/build_info.h>']
# Functions with one of these qualifiers are skipped.
_SKIP_FUNCTION_WITH_QUALIFIERS = frozenset(['inline', 'static'])
def __init__(self):
"""Construct a wrapper generator object.
"""
self.program_name = os.path.basename(sys.argv[0])
# To be populated in a derived class
self.functions = {} #type: Dict[str, FunctionInfo]
self._function_guards = {} #type: Dict[str, str]
# Preprocessor symbol used as a guard against multiple inclusion in the
# header. Must be set before writing output to a header.
# Not used when writing .c output.
self.header_guard = None #type: Optional[str]
def _write_prologue(self, out: typing_util.Writable, header: bool) -> None:
"""Write the prologue of a C file.
This includes a description comment and some include directives.
"""
prologue = strip_indentation(f'''
/* Automatically generated by {self.program_name}, do not edit! */
/* Copyright The Mbed TLS Contributors
* SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
*/
''')
if header:
prologue += strip_indentation(f'''
#ifndef {self.header_guard}
#define {self.header_guard}
#ifdef __cplusplus
extern "C" {{
#endif
''')
if not header:
# On Mingw-w64, force the use of a C99-compliant printf() and friends.
# This is necessary on older versions of Mingw and/or Windows runtimes
# where snprintf does not always zero-terminate the buffer, and does
# not support formats such as "%zu" for size_t and "%lld" for long long.
prologue += strip_indentation(f'''
#if !defined(__USE_MINGW_ANSI_STDIO)
#define __USE_MINGW_ANSI_STDIO 1
#endif
''')
for include in self._INCLUDES:
prologue += "#include {}\n".format(include)
# Make certain there is an empty line at the end of this section.
prologue += '\n' if self._INCLUDES else '\n\n'
out.write(prologue)
def _write_epilogue(self, out: typing_util.Writable, header: bool) -> None:
"""Write the epilogue of a C file."""
epilogue = ""
if header:
epilogue += strip_indentation(f'''
#ifdef __cplusplus
}}
#endif
#endif /* {self.header_guard} */
''')
epilogue += ('/* End of automatically generated file. */\n')
out.write(epilogue)
def _wrapper_function_name(self, original_name: str) -> str:
"""The name of the wrapper function.
By default, this adds a suffix.
"""
return (self._WRAPPER_NAME_PREFIX +
original_name +
self._WRAPPER_NAME_SUFFIX)
def _wrapper_declaration_start(self,
function: FunctionInfo,
wrapper_name: str) -> str:
"""The beginning of the wrapper function declaration.
This ends just before the opening parenthesis of the argument list.
This is a string containing at least the return type and the
function name. It may start with additional qualifiers or attributes
such as `static`, `__attribute__((...))`, etc.
"""
return c_declare(function.return_type, wrapper_name, '')
def _argument_name(self,
function_name: str,
num: int,
arg: ArgumentInfo) -> str:
"""Name to use for the given argument in the wrapper function.
Argument numbers count from 0.
"""
name = 'arg' + str(num)
if arg.name:
name += '_' + arg.name
return name
def _wrapper_declaration_argument(self,
function_name: str,
num: int, name: str,
arg: ArgumentInfo) -> str:
"""One argument definition in the wrapper function declaration.
Argument numbers count from 0.
"""
return c_declare(arg.type, name, arg.suffix)
def _underlying_function_name(self, function: FunctionInfo) -> str:
"""The name of the underlying function.
By default, this is the name of the wrapped function.
"""
return function.name
def _return_variable_name(self, function: FunctionInfo) -> str:
"""The name of the variable that will contain the return value."""
return 'retval'
def _write_function_call(self, out: typing_util.Writable,
function: FunctionInfo,
argument_names: List[str]) -> None:
"""Write the call to the underlying function.
"""
# Note that the function name is in parentheses, to avoid calling
# a function-like macro with the same name, since in typical usage
# there is a function-like macro with the same name which is the
# wrapper.
call = '({})({})'.format(self._underlying_function_name(function),
', '.join(argument_names))
if function.returns_void():
out.write(' {};\n'.format(call))
else:
ret_name = self._return_variable_name(function)
ret_decl = c_declare(function.return_type, ret_name, '')
out.write(' {} = {};\n'.format(ret_decl, call))
def _write_function_return(self, out: typing_util.Writable,
function: FunctionInfo,
if_void: bool = False) -> None:
"""Write a return statement.
If the function returns void, only write a statement if if_void is true.
"""
if function.returns_void():
if if_void:
out.write(' return;\n')
else:
ret_name = self._return_variable_name(function)
out.write(' return {};\n'.format(ret_name))
def _write_function_body(self, out: typing_util.Writable,
function: FunctionInfo,
argument_names: List[str]) -> None:
"""Write the body of the wrapper code for the specified function.
"""
self._write_function_call(out, function, argument_names)
self._write_function_return(out, function)
def _skip_function(self, function: FunctionInfo) -> bool:
"""Whether to skip this function.
By default, static or inline functions are skipped.
"""
if not self._SKIP_FUNCTION_WITH_QUALIFIERS.isdisjoint(function.qualifiers):
return True
return False
def _function_guard(self, function: FunctionInfo) -> Optional[str]:
"""A preprocessor condition for this function.
The wrapper will be guarded with `#if` on this condition, if not None.
"""
return self._function_guards.get(function.name)
def _wrapper_info(self, function: FunctionInfo) -> Optional[WrapperInfo]:
"""Information about the wrapper for one function.
Return None if the function should be skipped.
"""
if self._skip_function(function):
return None
argument_names = [self._argument_name(function.name, num, arg)
for num, arg in enumerate(function.arguments)]
return WrapperInfo(
argument_names=argument_names,
guard=self._function_guard(function),
wrapper_name=self._wrapper_function_name(function.name),
)
def _write_function_prototype(self, out: typing_util.Writable,
function: FunctionInfo,
wrapper: WrapperInfo,
header: bool) -> None:
"""Write the prototype of a wrapper function.
If header is true, write a function declaration, with a semicolon at
the end. Otherwise just write the prototype, intended to be followed
by the function's body.
"""
declaration_start = self._wrapper_declaration_start(function,
wrapper.wrapper_name)
arg_indent = ' '
terminator = ';\n' if header else '\n'
if function.arguments:
out.write(declaration_start + '(\n')
for num in range(len(function.arguments)):
arg_def = self._wrapper_declaration_argument(
function.name,
num, wrapper.argument_names[num], function.arguments[num])
arg_terminator = \
(')' + terminator if num == len(function.arguments) - 1 else
',\n')
out.write(arg_indent + arg_def + arg_terminator)
else:
out.write(declaration_start + '(void)' + terminator)
def _write_c_function(self, out: typing_util.Writable,
function: FunctionInfo) -> None:
"""Write wrapper code for one function.
Do nothing if the function is skipped.
"""
wrapper = self._wrapper_info(function)
if wrapper is None:
return
out.write('/* Wrapper for {} */\n'.format(function.name))
if wrapper.guard is not None:
out.write('#if {}\n'.format(wrapper.guard))
self._write_function_prototype(out, function, wrapper, False)
out.write('{\n')
self._write_function_body(out, function, wrapper.argument_names)
out.write('}\n')
if wrapper.guard is not None:
out.write('#endif /* {} */\n'.format(wrapper.guard))
out.write('\n')
def _write_h_function_declaration(self, out: typing_util.Writable,
function: FunctionInfo,
wrapper: WrapperInfo) -> None:
"""Write the declaration of one wrapper function.
"""
self._write_function_prototype(out, function, wrapper, True)
def _write_h_macro_definition(self, out: typing_util.Writable,
function: FunctionInfo,
wrapper: WrapperInfo) -> None:
"""Write the macro definition for one wrapper.
"""
arg_list = ', '.join(wrapper.argument_names)
out.write('#define {function_name}({args}) \\\n {wrapper_name}({args})\n'
.format(function_name=function.name,
wrapper_name=wrapper.wrapper_name,
args=arg_list))
def _write_h_function(self, out: typing_util.Writable,
function: FunctionInfo) -> None:
"""Write the complete header content for one wrapper.
This is the declaration of the wrapper function, and the
definition of a function-like macro that calls the wrapper function.
Do nothing if the function is skipped.
"""
wrapper = self._wrapper_info(function)
if wrapper is None:
return
if wrapper.guard is not None:
out.write('#if {}\n'.format(wrapper.guard))
self._write_h_function_declaration(out, function, wrapper)
self._write_h_macro_definition(out, function, wrapper)
if wrapper.guard is not None:
out.write('#endif /* {} */\n'.format(wrapper.guard))
out.write('\n')
def write_c_file(self, filename: str) -> None:
"""Output a whole C file containing function wrapper definitions."""
with open(filename, 'w', encoding='utf-8') as out:
self._write_prologue(out, False)
for name in sorted(self.functions):
self._write_c_function(out, self.functions[name])
self._write_epilogue(out, False)
def _header_guard_from_file_name(self, filename: str) -> str:
"""Preprocessor symbol used as a guard against multiple inclusion."""
# Heuristic to strip irrelevant leading directories
filename = re.sub(r'.*include[\\/]', r'', filename)
return re.sub(r'[^0-9A-Za-z]', r'_', filename, flags=re.A).upper()
def write_h_file(self, filename: str) -> None:
"""Output a header file with function wrapper declarations and macro definitions."""
self.header_guard = self._header_guard_from_file_name(filename)
with open(filename, 'w', encoding='utf-8') as out:
self._write_prologue(out, True)
for name in sorted(self.functions):
self._write_h_function(out, self.functions[name])
self._write_epilogue(out, True)
class UnknownTypeForPrintf(Exception):
"""Exception raised when attempting to generate code that logs a value of an unknown type."""
def __init__(self, typ: str) -> None:
super().__init__("Unknown type for printf format generation: " + typ)
class Logging(Base):
"""Generate wrapper functions that log the inputs and outputs."""
def __init__(self) -> None:
"""Construct a wrapper generator including logging of inputs and outputs.
Log to stdout by default. Call `set_stream` to change this.
"""
super().__init__()
self.stream = 'stdout'
def set_stream(self, stream: str) -> None:
"""Set the stdio stream to log to.
Call this method before calling `write_c_output` or `write_h_output`.
"""
self.stream = stream
def _write_prologue(self, out: typing_util.Writable, header: bool) -> None:
super()._write_prologue(out, header)
if not header:
out.write("""
#if defined(MBEDTLS_FS_IO) && defined(MBEDTLS_TEST_HOOKS)
#include <stdio.h>
#include <inttypes.h>
#include <mbedtls/platform.h> // for mbedtls_fprintf
#endif /* defined(MBEDTLS_FS_IO) && defined(MBEDTLS_TEST_HOOKS) */
""")
_PRINTF_SIMPLE_FORMAT = {
'int': '%d',
'long': '%ld',
'long long': '%lld',
'size_t': '%zu',
'unsigned': '0x%08x',
'unsigned int': '0x%08x',
'unsigned long': '0x%08lx',
'unsigned long long': '0x%016llx',
}
def _printf_simple_format(self, typ: str) -> Optional[str]:
"""Use this printf format for a value of typ.
Return None if values of typ need more complex handling.
"""
return self._PRINTF_SIMPLE_FORMAT.get(typ)
_PRINTF_TYPE_CAST = {
'int32_t': 'int',
'uint32_t': 'unsigned',
'uint64_t': 'unsigned long long',
} #type: Dict[str, str]
def _printf_type_cast(self, typ: str) -> Optional[str]:
"""Cast values of typ to this type before passing them to printf.
Return None if values of the given type do not need a cast.
"""
return self._PRINTF_TYPE_CAST.get(typ)
_POINTER_TYPE_RE = re.compile(r'\s*\*\Z')
def _printf_parameters(self, typ: str, var: str) -> Tuple[str, List[str]]:
"""The printf format and arguments for a value of type typ stored in var.
"""
expr = var
base_type = typ
# For outputs via a pointer, get the value that has been written.
# Note: we don't support pointers to pointers here.
pointer_match = self._POINTER_TYPE_RE.search(base_type)
if pointer_match:
base_type = base_type[:pointer_match.start(0)]
expr = '*({})'.format(expr)
# Maybe cast the value to a standard type.
cast_to = self._printf_type_cast(base_type)
if cast_to is not None:
expr = '({}) {}'.format(cast_to, expr)
base_type = cast_to
# Try standard types.
fmt = self._printf_simple_format(base_type)
if fmt is not None:
return '{}={}'.format(var, fmt), [expr]
raise UnknownTypeForPrintf(typ)
def _write_function_logging(self, out: typing_util.Writable,
function: FunctionInfo,
argument_names: List[str]) -> None:
"""Write code to log the function's inputs and outputs."""
formats, values = '%s', ['"' + function.name + '"']
for arg_info, arg_name in zip(function.arguments, argument_names):
fmt, vals = self._printf_parameters(arg_info.type, arg_name)
if fmt:
formats += ' ' + fmt
values += vals
if not function.returns_void():
ret_name = self._return_variable_name(function)
fmt, vals = self._printf_parameters(function.return_type, ret_name)
if fmt:
formats += ' ' + fmt
values += vals
out.write("""\
#if defined(MBEDTLS_FS_IO) && defined(MBEDTLS_TEST_HOOKS)
if ({stream}) {{
mbedtls_fprintf({stream}, "{formats}\\n",
{values});
}}
#endif /* defined(MBEDTLS_FS_IO) && defined(MBEDTLS_TEST_HOOKS) */
"""
.format(stream=self.stream,
formats=formats,
values=', '.join(values)))
def _write_function_body(self, out: typing_util.Writable,
function: FunctionInfo,
argument_names: List[str]) -> None:
"""Write the body of the wrapper code for the specified function.
"""
self._write_function_call(out, function, argument_names)
self._write_function_logging(out, function, argument_names)
self._write_function_return(out, function)

View File

@@ -0,0 +1,25 @@
""" PSA Buffer utility data-class.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
class BufferParameter:
"""Description of an input or output buffer parameter sequence to a PSA function."""
#pylint: disable=too-few-public-methods
def __init__(self, i: int, is_output: bool,
buffer_name: str, size_name: str) -> None:
"""Initialize the parameter information.
i is the index of the function argument that is the pointer to the buffer.
The size is argument i+1. For a variable-size output, the actual length
goes in argument i+2.
buffer_name and size_names are the names of arguments i and i+1.
This class does not yet help with the output length.
"""
self.index = i
self.buffer_name = buffer_name
self.size_name = size_name
self.is_output = is_output

View File

@@ -0,0 +1,27 @@
"""Generate wrapper functions for PSA function calls.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
from typing import List, Optional
from .psa_wrapper import PSAWrapper, PSALoggingWrapper
class PSATestWrapper(PSAWrapper):
"""Generate a C source file containing wrapper functions for PSA Crypto API calls."""
_WRAPPER_NAME_PREFIX = 'mbedtls_test_wrap_'
_WRAPPER_NAME_SUFFIX = ''
_PSA_WRAPPER_INCLUDES = ['<psa/crypto.h>',
'<test/memory.h>',
'<test/psa_crypto_helpers.h>',
'<test/psa_test_wrappers.h>']
class PSALoggingTestWrapper(PSATestWrapper, PSALoggingWrapper):
"""Generate a C source file containing wrapper functions that log PSA Crypto API calls."""
def __init__(self, out_h_f: str, out_c_f: str, stream: str,
in_headers: Optional[List[str]] = None) -> None:
super().__init__(out_h_f, out_c_f, stream, in_headers)

View File

@@ -0,0 +1,287 @@
"""Generate wrapper functions for PSA function calls.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import itertools
import os
from typing import Iterable, Iterator, List, Optional, Tuple
from .. import build_tree
from .. import c_parsing_helper
from .. import c_wrapper_generator
from .. import typing_util
from .psa_buffer import BufferParameter
class PSAWrapperConfiguration:
"""Configuration data class for PSA Wrapper."""
#pylint: disable=too-few-public-methods
def __init__(self) -> None:
self.cpp_guards = [
"MBEDTLS_PSA_CRYPTO_C",
"MBEDTLS_TEST_HOOKS",
"!RECORD_PSA_STATUS_COVERAGE_LOG",
]
self.skipped_functions = frozenset([
'mbedtls_psa_external_get_random', # not a library function
'psa_get_key_domain_parameters', # client-side function
'psa_get_key_slot_number', # client-side function
'psa_key_derivation_verify_bytes', # not implemented yet
'psa_key_derivation_verify_key', # not implemented yet
'psa_set_key_domain_parameters', # client-side function
])
self.skipped_argument_types = frozenset([
# PAKE stuff: not implemented yet
'psa_crypto_driver_pake_inputs_t *',
'psa_pake_cipher_suite_t *',
])
self.function_guards = {
'mbedtls_psa_register_se_key': 'defined(MBEDTLS_PSA_CRYPTO_SE_C)',
'mbedtls_psa_inject_entropy': 'defined(MBEDTLS_PSA_INJECT_ENTROPY)',
'mbedtls_psa_external_get_random': 'defined(MBEDTLS_PSA_CRYPTO_EXTERNAL_RNG)',
'mbedtls_psa_platform_get_builtin_key': 'defined(MBEDTLS_PSA_CRYPTO_BUILTIN_KEYS)',
'psa_crypto_driver_pake_get_cipher_suite' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_crypto_driver_pake_get_password' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_crypto_driver_pake_get_password_len' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_crypto_driver_pake_get_peer' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_crypto_driver_pake_get_peer_len' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_crypto_driver_pake_get_user' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_crypto_driver_pake_get_user_len' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_abort' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_get_implicit_key' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_input' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_output' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_set_password_key' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_set_peer' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_set_role' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_set_user' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
'psa_pake_setup' : 'defined(PSA_WANT_ALG_SOME_PAKE)',
}
class PSAWrapper(c_wrapper_generator.Base):
"""Generate a C source file containing wrapper functions for PSA Crypto API calls."""
_WRAPPER_NAME_PREFIX = 'mbedtls_test_wrap_'
_WRAPPER_NAME_SUFFIX = ''
_PSA_WRAPPER_INCLUDES = ['<psa/crypto.h>']
_DEFAULT_IN_HEADERS = ['crypto.h', 'crypto_extra.h']
def __init__(self,
out_h_f: str,
out_c_f: str,
in_headers: Optional[List[str]] = None,
config: PSAWrapperConfiguration = PSAWrapperConfiguration()) -> None:
super().__init__()
self.out_c_f = out_c_f
self.out_h_f = out_h_f
self.project_root = build_tree.guess_project_root()
self.read_config(config)
self.read_headers(in_headers)
def read_config(self, cfg: PSAWrapperConfiguration)-> None:
"""Configure instance's parameters from a user provided config."""
self._cpp_guards = PSAWrapper.parse_def_guards(cfg.cpp_guards)
self._skip_functions = cfg.skipped_functions
self._function_guards.update(cfg.function_guards)
self._not_implemented = cfg.skipped_argument_types
def read_headers(self, headers: Optional[List[str]]) -> None:
"""Reads functions to be wrapped from source header files into self.functions."""
self.in_headers = self._DEFAULT_IN_HEADERS if headers is None else headers
for header_name in self.in_headers:
header_path = self.rel_path(header_name)
c_parsing_helper.read_function_declarations(self.functions, header_path)
def rel_path(self, filename: str, path_list: Optional[List[str]] = None) -> str:
"""Return the estimated path in relationship to the project_root.
The method allows overriding the targetted sub-directory.
Currently the default is set to project_root/include/psa."""
if path_list is None:
path_list = ['include', 'psa']
return os.path.join(self.project_root, *path_list, filename)
# Utility Methods
@staticmethod
def parse_def_guards(def_list: Iterable[str])-> str:
""" Create define guards.
Convert an input list of into a C preprocessor
expression of defined() && !defined() syntax string."""
output = ""
dl = [("defined({})".format(n) if n[0] != "!" else
"!defined({})".format(n[1:]))
for n in def_list]
# Split the list in chunks of 2 and add new lines
for i in range(0, len(dl), 2):
output += "{} && {} && \\".format(dl[i], dl[i+1]) + "\n "\
if i+2 <= len(dl) else dl[i]
return output
@staticmethod
def _detect_buffer_parameters(arguments: List[c_parsing_helper.ArgumentInfo],
argument_names: List[str]) -> Iterator[BufferParameter]:
"""Detect function arguments that are buffers (pointer, size [,length])."""
types = ['' if arg.suffix else arg.type for arg in arguments]
# pairs = list of (type_of_arg_N, type_of_arg_N+1)
# where each type_of_arg_X is the empty string if the type is an array
# or there is no argument X.
pairs = enumerate(itertools.zip_longest(types, types[1:], fillvalue=''))
for i, t01 in pairs:
if (t01[0] == 'const uint8_t *' or t01[0] == 'uint8_t *') and \
t01[1] == 'size_t':
yield BufferParameter(i, not t01[0].startswith('const '),
argument_names[i], argument_names[i+1])
@staticmethod
def _parameter_should_be_copied(function_name: str,
_buffer_name: Optional[str]) -> bool:
"""Whether the specified buffer argument to a PSA function should be copied.
"""
# False-positives that do not need buffer copying
if function_name in ('mbedtls_psa_inject_entropy',
'psa_crypto_driver_pake_get_password',
'psa_crypto_driver_pake_get_user',
'psa_crypto_driver_pake_get_peer'):
return False
return True
@staticmethod
def _poison_wrap(param: BufferParameter, poison: bool,
ident_lv: int = 1) -> str:
"""Returns a call to MBEDTLS_TEST_MEMORY_[UN]POISON.
The output is prefixed with MBEDTLS_TEST_MEMORY_ followed by POISON/UNPOISON
and the input parameter arguments (name, length)
"""
return "{}MBEDTLS_TEST_MEMORY_{}({}, {});\n".format(
(ident_lv * 4) * ' ',
'POISON' if poison else 'UNPOISON',
param.buffer_name, param.size_name)
def _poison_multi_write(self,
out: typing_util.Writable,
buffer_parameters: List['BufferParameter'],
poison: bool) -> None:
"""Write poisoning or unpoisoning code for the buffer parameters.
Write poisoning code if poison is true, unpoisoning code otherwise.
"""
if not buffer_parameters:
return
out.write('#if !defined(MBEDTLS_PSA_ASSUME_EXCLUSIVE_BUFFERS)\n')
for param in buffer_parameters:
out.write(self._poison_wrap(param, poison))
out.write('#endif /* !defined(MBEDTLS_PSA_ASSUME_EXCLUSIVE_BUFFERS) */\n')
# Override parent's methods
def _write_function_call(self, out: typing_util.Writable,
function: c_wrapper_generator.FunctionInfo,
argument_names: List[str]) -> None:
buffer_parameters = list(
param
for param in self._detect_buffer_parameters(function.arguments,
argument_names)
if self._parameter_should_be_copied(function.name,
function.arguments[param.index].name))
self._poison_multi_write(out, buffer_parameters, True)
super()._write_function_call(out, function, argument_names)
self._poison_multi_write(out, buffer_parameters, False)
def _skip_function(self, function: c_wrapper_generator.FunctionInfo) -> bool:
if function.return_type != 'psa_status_t':
return True
if function.name in self._skip_functions:
return True
return False
def _return_variable_name(self,
function: c_wrapper_generator.FunctionInfo) -> str:
"""The name of the variable that will contain the return value."""
if function.return_type == 'psa_status_t':
return 'status'
return super()._return_variable_name(function)
def _write_prologue(self, out: typing_util.Writable, header: bool) -> None:
super()._write_prologue(out, header)
prologue = []
if self._cpp_guards:
prologue.append("#if {}".format(self._cpp_guards))
prologue.append('')
for include in self._PSA_WRAPPER_INCLUDES:
prologue.append("#include {}".format(include))
# Make certain there is an empty line at the end of this section.
for i in [-1, -2]:
if prologue[i] != '':
prologue.append('')
out.write("\n".join(prologue))
def _write_epilogue(self, out: typing_util.Writable, header: bool) -> None:
if self._cpp_guards:
out.write("#endif /* {} */\n\n".format(self._cpp_guards))
super()._write_epilogue(out, header)
class PSALoggingWrapper(PSAWrapper, c_wrapper_generator.Logging):
"""Generate a C source file containing wrapper functions that log PSA Crypto API calls."""
def __init__(self, #pylint: disable=too-many-arguments
stream: str,
out_h_f: str,
out_c_f: str,
in_headers: Optional[List[str]] = None,
config: PSAWrapperConfiguration = PSAWrapperConfiguration()) -> None:
super().__init__(out_h_f, out_c_f, in_headers, config)
self.set_stream(stream)
_PRINTF_TYPE_CAST = c_wrapper_generator.Logging._PRINTF_TYPE_CAST.copy()
_PRINTF_TYPE_CAST.update({
'mbedtls_svc_key_id_t': 'unsigned',
'psa_algorithm_t': 'unsigned',
'psa_drv_slot_number_t': 'unsigned long long',
'psa_key_derivation_step_t': 'int',
'psa_key_id_t': 'unsigned',
'psa_key_slot_number_t': 'unsigned long long',
'psa_key_lifetime_t': 'unsigned',
'psa_key_type_t': 'unsigned',
'psa_key_usage_flags_t': 'unsigned',
'psa_pake_role_t': 'int',
'psa_pake_step_t': 'int',
'psa_status_t': 'int',
})
def _printf_parameters(self, typ: str, var: str) -> Tuple[str, List[str]]:
if typ.startswith('const '):
typ = typ[6:]
if typ == 'uint8_t *':
# Skip buffers
return '', []
if typ.endswith('operation_t *'):
return '', []
if typ in self._not_implemented:
return '', []
if typ == 'psa_key_attributes_t *':
return (var + '={id=%u, lifetime=0x%08x, type=0x%08x, bits=%u, alg=%08x, usage=%08x}',
['(unsigned) psa_get_key_{}({})'.format(field, var)
for field in ['id', 'lifetime', 'type', 'bits', 'algorithm', 'usage_flags']])
return super()._printf_parameters(typ, var)

View File

@@ -0,0 +1,170 @@
"""Discover all the test cases (unit tests and SSL tests)."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import glob
import os
import re
import subprocess
import sys
from . import build_tree
class ScriptOutputError(ValueError):
"""A kind of ValueError that indicates we found
the script doesn't list test cases in an expected
pattern.
"""
@property
def script_name(self):
return super().args[0]
@property
def idx(self):
return super().args[1]
@property
def line(self):
return super().args[2]
class Results:
"""Store file and line information about errors or warnings in test suites."""
def __init__(self, options):
self.errors = 0
self.warnings = 0
self.ignore_warnings = options.quiet
def error(self, file_name, line_number, fmt, *args):
sys.stderr.write(('{}:{}:ERROR:' + fmt + '\n').
format(file_name, line_number, *args))
self.errors += 1
def warning(self, file_name, line_number, fmt, *args):
if not self.ignore_warnings:
sys.stderr.write(('{}:{}:Warning:' + fmt + '\n')
.format(file_name, line_number, *args))
self.warnings += 1
class TestDescriptionExplorer:
"""An iterator over test cases with descriptions.
The test cases that have descriptions are:
* Individual unit tests (entries in a .data file) in test suites.
* Individual test cases in ssl-opt.sh.
This is an abstract class. To use it, derive a class that implements
the process_test_case method, and call walk_all().
"""
def process_test_case(self, per_file_state,
file_name, line_number, description):
"""Process a test case.
per_file_state: an object created by new_per_file_state() at the beginning
of each file.
file_name: a relative path to the file containing the test case.
line_number: the line number in the given file.
description: the test case description as a byte string.
"""
raise NotImplementedError
def new_per_file_state(self):
"""Return a new per-file state object.
The default per-file state object is None. Child classes that require per-file
state may override this method.
"""
#pylint: disable=no-self-use
return None
def walk_test_suite(self, data_file_name):
"""Iterate over the test cases in the given unit test data file."""
in_paragraph = False
descriptions = self.new_per_file_state() # pylint: disable=assignment-from-none
with open(data_file_name, 'rb') as data_file:
for line_number, line in enumerate(data_file, 1):
line = line.rstrip(b'\r\n')
if not line:
in_paragraph = False
continue
if line.startswith(b'#'):
continue
if not in_paragraph:
# This is a test case description line.
self.process_test_case(descriptions,
data_file_name, line_number, line)
in_paragraph = True
def collect_from_script(self, script_name):
"""Collect the test cases in a script by calling its listing test cases
option"""
descriptions = self.new_per_file_state() # pylint: disable=assignment-from-none
listed = subprocess.check_output(['sh', script_name, '--list-test-cases'])
# Assume test file is responsible for printing identical format of
# test case description between --list-test-cases and its OUTCOME.CSV
#
# idx indicates the number of test case since there is no line number
# in the script for each test case.
for idx, line in enumerate(listed.splitlines()):
# We are expecting the script to list the test cases in
# `<suite_name>;<description>` pattern.
script_outputs = line.split(b';', 1)
if len(script_outputs) == 2:
suite_name, description = script_outputs
else:
raise ScriptOutputError(script_name, idx, line.decode("utf-8"))
self.process_test_case(descriptions,
suite_name.decode('utf-8'),
idx,
description.rstrip())
@staticmethod
def collect_test_directories():
"""Get the relative path for the TLS and Crypto test directories."""
project_root = build_tree.guess_project_root()
if build_tree.looks_like_mbedtls_root(project_root) and not build_tree.is_mbedtls_3_6():
directories = [os.path.join(project_root, 'tests'),
os.path.join(project_root, 'tf-psa-crypto', 'tests')]
else:
directories = [os.path.join(project_root, 'tests')]
directories = [os.path.relpath(p) for p in directories]
return directories
def walk_all(self):
"""Iterate over all named test cases."""
test_directories = self.collect_test_directories()
for directory in test_directories:
for data_file_name in glob.glob(os.path.join(directory, 'suites',
'*.data')):
self.walk_test_suite(data_file_name)
for sh_file in ['ssl-opt.sh', 'compat.sh']:
sh_file = os.path.join(directory, sh_file)
if os.path.isfile(sh_file):
self.collect_from_script(sh_file)
class TestDescriptions(TestDescriptionExplorer):
"""Collect the available test cases."""
def __init__(self):
super().__init__()
self.descriptions = set()
def process_test_case(self, _per_file_state,
file_name, _line_number, description):
"""Record an available test case."""
base_name = re.sub(r'\.[^.]*$', '', re.sub(r'.*/', '', file_name))
key = ';'.join([base_name, description.decode('utf-8')])
self.descriptions.add(key)
def collect_available_test_cases():
"""Collect the available test cases."""
explorer = TestDescriptions()
explorer.walk_all()
return sorted(explorer.descriptions)

View File

@@ -0,0 +1,297 @@
"""Generate C preprocessor code to check for bad configurations.
The headers are meant to be included in a specific way in PROJECT_config.c.
See framework/docs/architecture/config-check-framework.md for information.
"""
## Copyright The Mbed TLS Contributors
## SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import enum
import os
import posixpath
import re
import sys
import textwrap
import typing
from typing import Iterator, List, Optional
from . import build_tree
from . import typing_util
class Position(enum.Enum):
BEFORE = 0 # Before build_info.h
USER = 1 # Just after reading PROJECT_CONFIG_FILE (*config.h) and PROJECT_USER_CONFIG_FILE
FINAL = 2 # After *adjust*.h and the rest of build_info.h
class Checker:
"""Description of checks for one option.
By default, this class triggers an error if the option is set after
reading the user configuration. To change the behavior, override
the methods `before`, `user` and `final` as needed.
"""
def __init__(self, name: str, suggestion: str = '') -> None:
"""Construct a checker for the given preprocessor macro name.
If suggestion is given, it is appended to the error message.
It should be a short sentence intended for human readers.
This sentence follows a sentence like "<macro_name> is not
a valid configuration option".
"""
self.name = name
self.suggestion = suggestion
def _basic_message(self) -> str:
"""The first sentence of the message to display on error.
It should end with a full stop or other sentence-ending punctuation.
"""
return f'{self.name} is not a valid configuration option.'
def message(self) -> str:
"""The message to display on error."""
message = self._basic_message()
if self.suggestion:
message += ' Suggestion: ' + self.suggestion
return message
def _quoted_message(self) -> str:
"""Quote message() in double quotes. Useful for #error directives."""
return ('"' +
str.replace(str.replace(self.message(),
'\\', '\\\\'),
'"', '\\"') +
'"')
def before(self, _prefix: str) -> str:
"""C code to inject before including the config."""
#pylint: disable=no-self-use
# Derived classes will add content where needed.
return ''
def user(self, _prefix: str) -> str:
"""C code to inject immediately after including the user config."""
return f'''
#if defined({self.name})
# error {self._quoted_message()}
#endif
'''
def final(self, _prefix: str) -> str:
"""C code to inject after finalizing the config."""
#pylint: disable=no-self-use
# Derived classes will add content where needed.
return ''
def code(self, position: Position, prefix: str) -> str:
"""C code to inject at the given position.
Use the given prefix for auxiliary macro names.
"""
methods = {
Position.BEFORE: self.before,
Position.USER: self.user,
Position.FINAL: self.final,
}
method = methods[position]
snippet = method(prefix)
return textwrap.dedent(snippet)
class Internal(Checker):
"""Checker for an internal-only option."""
class SubprojectInternal(Checker):
"""Checker for an internal macro of a subproject."""
# must be overridden in child classes
SUBPROJECT = None #type: Optional[str]
def __init__(self, name: str, suggestion: str = '') -> None:
super().__init__(name, suggestion)
if self.SUBPROJECT is None:
raise ValueError('No subproject specified for ' + name)
self.subproject = self.SUBPROJECT #type: str
def _basic_message(self) -> str:
return f'{self.name} is an internal macro of {self.subproject} and may not be configured.'
def before(self, prefix: str) -> str:
return f'''
#if defined({self.name})
# define {prefix}_WASSET_{self.name} 1
#else
# define {prefix}_WASSET_{self.name} 0
#endif
#undef {self.name}
'''
def user(self, prefix: str) -> str:
return f'''
#if defined({self.name})
# error {self._quoted_message()}
#endif
#if {prefix}_WASSET_{self.name}
# define {self.name}
#endif
#undef {prefix}_WASSET_{self.name}
'''
class SubprojectOption(SubprojectInternal):
"""Checker for a configuration option of a subproject."""
def _basic_message(self) -> str:
return f'{self.name} must be configured in {self.subproject}.'
def user(self, prefix: str) -> str:
return f'''
#if defined({self.name}) && !{prefix}_WASSET_{self.name}
# error {self._quoted_message()}
#endif
#if {prefix}_WASSET_{self.name}
# define {self.name}
#endif
#undef {prefix}_WASSET_{self.name}
'''
class Removed(Checker):
"""Checker for an option that has been removed."""
def __init__(self, name: str, version: str, suggestion: str = '') -> None:
super().__init__(name, suggestion)
self.version = version
def _basic_message(self) -> str:
"""The first sentence of the message to display on error.
It should end with a full stop or other sentence-ending punctuation.
"""
return f'{self.name} was removed in {self.version}.'
def user(self, prefix: str) -> str:
"""C code to inject immediately after including the user config."""
# A removed option is forbidden, just like an internal option.
# But since we're checking a macro that is not defined anywhere,
# we need to tell check_names.py that this is a false positive.
code = super().user(prefix)
return re.sub(rf'^ *# *\w+.*\b{self.name}\b.*$',
lambda m: m.group(0) + ' //no-check-names',
code, flags=re.M)
class BranchData(typing.NamedTuple):
"""The relevant aspects of the configuration on a branch."""
# Subdirectory where the generated headers will be located.
header_directory: str
# Prefix used for the generated headers' basename.
header_prefix: str
# Prefix used for C preprocessor macros.
project_cpp_prefix: str
# Options to check
checkers: List[Checker]
class HeaderGenerator:
"""Generate a header to include before or after the user config."""
def __init__(self, branch_data: BranchData, position: Position) -> None:
self.branch_data = branch_data
self.position = position
self.prefix = branch_data.project_cpp_prefix + '_CONFIG_CHECK'
self.bypass_checks = self.prefix + '_BYPASS'
def write_stanza(self, out: typing_util.Writable, checker: Checker) -> None:
"""Write the part of the output corresponding to one config option."""
code = checker.code(self.position, self.prefix)
out.write(code)
def write_content(self, out: typing_util.Writable) -> None:
"""Write the output for all config options to be processed."""
for checker in self.branch_data.checkers:
self.write_stanza(out, checker)
def output_file_name(self) -> str:
"""The base name of the output file."""
return ''.join([self.branch_data.header_prefix,
'config_check_',
self.position.name.lower(),
'.h'])
def write(self, directory: str) -> None:
"""Write the whole output file."""
basename = self.output_file_name()
with open(os.path.join(directory, basename), 'w') as out:
out.write(f'''\
/* {basename} (generated part of {self.branch_data.header_prefix}config.c). */
/* Automatically generated by {os.path.basename(sys.argv[0])}. Do not edit! */
#if !defined({self.bypass_checks}) //no-check-names
/* *INDENT-OFF* */
''')
self.write_content(out)
out.write(f'''
/* *INDENT-ON* */
#endif /* !defined({self.bypass_checks}) */ //no-check-names
/* End of automatically generated {basename} */
''')
def generate_header_files(branch_data: BranchData,
directory: str,
list_only: bool = False) -> Iterator[str]:
"""Generate the header files to include before and after *config.h."""
for position in Position:
generator = HeaderGenerator(branch_data, position)
# Make sure to output a path with / even on Windows, so that
# it can be consumed by tools such as CMake.
yield posixpath.join(directory, generator.output_file_name())
if not list_only:
generator.write(directory)
def main(branch_data: BranchData) -> None:
root = build_tree.guess_project_root()
# Is root the current directory? The safe default is no, so compare
# the paths, rather than calling `os.samefile()` which can have false
# positives and can fail in edge cases.
if root == os.getcwd():
# Be nice and use a relative path when it's simple to do so.
# (build_tree.guess_project_root() should probably do this, actually.)
# This is not only nice to humans, but also necessary for
# `make_generated_files.py --root DIR --check`: it calls
# this script with `--list` and expects a path that is relative
# to DIR, not an absolute path that is under the project root.
root = os.curdir
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--list', action='store_true',
help='List generated files and exit')
parser.add_argument('--list-for-cmake', action='store_true',
help='List generated files in CMake-friendly format and exit')
parser.add_argument('output_directory', metavar='DIR', nargs='?',
default=posixpath.join(root, branch_data.header_directory),
help='output file location (default: %(default)s)')
options = parser.parse_args()
list_only = options.list or options.list_for_cmake
output_files = list(generate_header_files(branch_data,
options.output_directory,
list_only=list_only))
if options.list_for_cmake:
sys.stdout.write(';'.join(output_files))
elif options.list:
for filename in output_files:
print(filename)

View File

@@ -0,0 +1,507 @@
"""Mbed TLS and PSA configuration file manipulation library
"""
## Copyright The Mbed TLS Contributors
## SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
##
import argparse
import os
import re
import shutil
import sys
from abc import ABCMeta
class Setting:
"""Representation of one Mbed TLS mbedtls_config.h or PSA crypto_config.h setting.
Fields:
* name: the symbol name ('MBEDTLS_xxx').
* value: the value of the macro. The empty string for a plain #define
with no value.
* active: True if name is defined, False if a #define for name is
present in mbedtls_config.h but commented out.
* section: the name of the section that contains this symbol.
* configfile: the representation of the configuration file where the setting is defined
"""
# pylint: disable=too-few-public-methods, too-many-arguments
def __init__(self, configfile, active, name, value='', section=None):
self.active = active
self.name = name
self.value = value
self.section = section
self.configfile = configfile
class Config:
"""Representation of the Mbed TLS and PSA configuration.
In the documentation of this class, a symbol is said to be *active*
if there is a #define for it that is not commented out, and *known*
if there is a #define for it whether commented out or not.
This class supports the following protocols:
* `name in config` is `True` if the symbol `name` is active, `False`
otherwise (whether `name` is inactive or not known).
* `config[name]` is the value of the macro `name`. If `name` is inactive,
raise `KeyError` (even if `name` is known).
* `config[name] = value` sets the value associated to `name`. `name`
must be known, but does not need to be set. This does not cause
name to become set.
"""
def __init__(self):
self.settings = {}
self.configfiles = []
def __contains__(self, name):
"""True if the given symbol is active (i.e. set).
False if the given symbol is not set, even if a definition
is present but commented out.
"""
return name in self.settings and self.settings[name].active
def all(self, *names):
"""True if all the elements of names are active (i.e. set)."""
return all(name in self for name in names)
def any(self, *names):
"""True if at least one symbol in names are active (i.e. set)."""
return any(name in self for name in names)
def known(self, name):
"""True if a #define for name is present, whether it's commented out or not."""
return name in self.settings
def __getitem__(self, name):
"""Get the value of name, i.e. what the preprocessor symbol expands to.
If name is not known, raise KeyError. name does not need to be active.
"""
return self.settings[name].value
def get(self, name, default=None):
"""Get the value of name. If name is inactive (not set), return default.
If a #define for name is present and not commented out, return
its expansion, even if this is the empty string.
If a #define for name is present but commented out, return default.
"""
if name in self:
return self.settings[name].value
else:
return default
def get_matching(self, regexs, only_enabled):
"""Get all symbols matching one of the regexs."""
if not regexs:
return
regex = re.compile('|'.join(regexs))
for setting in self.settings.values():
if regex.search(setting.name):
if setting.active or not only_enabled:
yield setting.name
def __setitem__(self, name, value):
"""If name is known, set its value.
If name is not known, raise KeyError.
"""
setting = self.settings[name]
if setting.value != value:
setting.configfile.modified = True
setting.value = value
def set(self, name, value=None):
"""Set name to the given value and make it active.
If value is None and name is already known, don't change its value.
If value is None and name is not known, set its value.
"""
if name in self.settings:
setting = self.settings[name]
if (value is not None and setting.value != value) or not setting.active:
setting.configfile.modified = True
if value is not None:
setting.value = value
setting.active = True
else:
configfile = self._get_configfile(name)
self.settings[name] = Setting(configfile, True, name, value=value)
configfile.modified = True
def unset(self, name):
"""Make name unset (inactive).
name remains known if it was known before.
"""
if name not in self.settings:
return
setting = self.settings[name]
# Check if modifying the config file
if setting.active:
setting.configfile.modified = True
setting.active = False
def adapt(self, adapter):
"""Run adapter on each known symbol and (de)activate it accordingly.
`adapter` must be a function that returns a boolean. It is called as
`adapter(name, value, active)` for each setting, where
`value` is the macro's expansion (possibly empty), and `active` is
`True` if `name` is set and `False` if `name` is known but unset.
If `adapter` returns `True`, then set `name` (i.e. make it active),
otherwise unset `name` (i.e. make it known but inactive).
"""
for setting in self.settings.values():
is_active = setting.active
setting.active = adapter(setting.name, setting.value,
setting.active)
# Check if modifying the config file
if setting.active != is_active:
setting.configfile.modified = True
def change_matching(self, regexs, enable):
"""Change all symbols matching one of the regexs to the desired state."""
if not regexs:
return
regex = re.compile('|'.join(regexs))
for setting in self.settings.values():
if regex.search(setting.name):
# Check if modifying the config file
if setting.active != enable:
setting.configfile.modified = True
setting.active = enable
def _get_configfile(self, name=None):
"""Get the representation of the configuration file name belongs to
If the configuration is spread among several configuration files, this
function may need to be overridden for the case of an unknown setting.
"""
if name and name in self.settings:
return self.settings[name].configfile
return self.configfiles[0]
def write(self, filename=None):
"""Write the whole configuration to the file(s) it was read from.
If filename is specified, write to this file(s) instead.
"""
for configfile in self.configfiles:
configfile.write(self.settings, filename)
def filename(self, name=None):
"""Get the name of the config file where the setting name is defined."""
return self._get_configfile(name).filename
def backup(self, suffix='.bak'):
"""Back up the configuration file."""
for configfile in self.configfiles:
configfile.backup(suffix)
def restore(self):
"""Restore the configuration file."""
for configfile in self.configfiles:
configfile.restore()
class ConfigFile(metaclass=ABCMeta):
"""Representation of a configuration file."""
def __init__(self, default_path, name, filename=None):
"""Check if the config file exists."""
if filename is None:
for candidate in default_path:
if os.path.lexists(candidate):
filename = candidate
break
if not os.path.lexists(filename):
raise FileNotFoundError(f'{name} configuration file not found: '
f'{filename if filename else default_path}')
self.filename = filename
self.templates = []
self.current_section = None
self.inclusion_guard = None
self.modified = False
self._backupname = None
self._own_backup = False
_define_line_regexp = (r'(?P<indentation>\s*)' +
r'(?P<commented_out>(//\s*)?)' +
r'(?P<define>#\s*define\s+)' +
r'(?P<name>\w+)' +
r'(?P<arguments>(?:\((?:\w|\s|,)*\))?)' +
r'(?P<separator>\s*)' +
r'(?P<value>.*)')
_ifndef_line_regexp = r'#ifndef (?P<inclusion_guard>\w+)'
_section_line_regexp = (r'\s*/?\*+\s*[\\@]name\s+SECTION:\s*' +
r'(?P<section>.*)[ */]*')
_config_line_regexp = re.compile(r'|'.join([_define_line_regexp,
_ifndef_line_regexp,
_section_line_regexp]))
def _parse_line(self, line):
"""Parse a line in the config file, save the templates representing the lines
and return the corresponding setting element.
"""
line = line.rstrip('\r\n')
m = re.match(self._config_line_regexp, line)
if m is None:
self.templates.append(line)
return None
elif m.group('section'):
self.current_section = m.group('section')
self.templates.append(line)
return None
elif m.group('inclusion_guard') and self.inclusion_guard is None:
self.inclusion_guard = m.group('inclusion_guard')
self.templates.append(line)
return None
else:
active = not m.group('commented_out')
name = m.group('name')
value = m.group('value')
if name == self.inclusion_guard and value == '':
# The file double-inclusion guard is not an option.
self.templates.append(line)
return None
template = (name,
m.group('indentation'),
m.group('define') + name +
m.group('arguments') + m.group('separator'))
self.templates.append(template)
return (active, name, value, self.current_section)
def parse_file(self):
"""Parse the whole file and return the settings."""
with open(self.filename, 'r', encoding='utf-8') as file:
for line in file:
setting = self._parse_line(line)
if setting is not None:
yield setting
self.current_section = None
#pylint: disable=no-self-use
def _format_template(self, setting, indent, middle):
"""Build a line for the config file for the given setting.
The line has the form "<indent>#define <name> <value>"
where <middle> is "#define <name> ".
"""
value = setting.value
if value is None:
value = ''
# Normally the whitespace to separate the symbol name from the
# value is part of middle, and there's no whitespace for a symbol
# with no value. But if a symbol has been changed from having a
# value to not having one, the whitespace is wrong, so fix it.
if value:
if middle[-1] not in '\t ':
middle += ' '
else:
middle = middle.rstrip()
return ''.join([indent,
'' if setting.active else '//',
middle,
value]).rstrip()
def write_to_stream(self, settings, output):
"""Write the whole configuration to output."""
for template in self.templates:
if isinstance(template, str):
line = template
else:
name, indent, middle = template
line = self._format_template(settings[name], indent, middle)
output.write(line + '\n')
def write(self, settings, filename=None):
"""Write the whole configuration to the file it was read from.
If filename is specified, write to this file instead.
"""
if filename is None:
filename = self.filename
# Not modified so no need to write to the file
if not self.modified and filename == self.filename:
return
with open(filename, 'w', encoding='utf-8') as output:
self.write_to_stream(settings, output)
def backup(self, suffix='.bak'):
"""Back up the configuration file.
If the backup file already exists, it is presumed to be the desired backup,
so don't make another backup.
"""
if self._backupname:
return
self._backupname = self.filename + suffix
if os.path.exists(self._backupname):
self._own_backup = False
else:
self._own_backup = True
shutil.copy(self.filename, self._backupname)
def restore(self):
"""Restore the configuration file.
Only delete the backup file if it was created earlier.
"""
if not self._backupname:
return
if self._own_backup:
shutil.move(self._backupname, self.filename)
else:
shutil.copy(self._backupname, self.filename)
self._backupname = None
class ConfigTool(metaclass=ABCMeta):
"""Command line config manipulation tool.
Custom parser options can be added by overriding 'custom_parser_options'.
"""
def __init__(self, default_file_path):
"""Create parser for config manipulation tool.
:param default_file_path: Default configuration file path
"""
self.parser = argparse.ArgumentParser(description="""
Configuration file manipulation tool.""")
self.subparsers = self.parser.add_subparsers(dest='command',
title='Commands')
self._common_parser_options(default_file_path)
self.custom_parser_options()
self.args = self.parser.parse_args()
self.config = Config() # Make the pylint happy
def add_adapter(self, name, function, description):
"""Creates a command in the tool for a configuration adapter."""
subparser = self.subparsers.add_parser(name, help=description)
subparser.set_defaults(adapter=function)
def _common_parser_options(self, default_file_path):
# pylint: disable=too-many-branches
"""Common parser options for config manipulation tool."""
self.parser.add_argument(
'--file', '-f',
help="""File to read (and modify if requested). Default: {}.
""".format(default_file_path))
self.parser.add_argument(
'--force', '-o',
action='store_true',
help="""For the set command, if SYMBOL is not present, add a definition for it.""")
self.parser.add_argument(
'--write', '-w',
metavar='FILE',
help="""File to write to instead of the input file.""")
parser_get = self.subparsers.add_parser(
'get',
help="""Find the value of SYMBOL and print it. Exit with
status 0 if a #define for SYMBOL is found, 1 otherwise.""")
parser_get.add_argument('symbol', metavar='SYMBOL')
parser_set = self.subparsers.add_parser(
'set',
help="""Set SYMBOL to VALUE. If VALUE is omitted, just uncomment
the #define for SYMBOL. Error out of a line defining
SYMBOL (commented or not) is not found, unless --force is passed. """)
parser_set.add_argument('symbol', metavar='SYMBOL')
parser_set.add_argument('value', metavar='VALUE', nargs='?', default='')
parser_set_all = self.subparsers.add_parser(
'set-all',
help="""Uncomment all #define whose name contains a match for REGEX.""")
parser_set_all.add_argument('regexs', metavar='REGEX', nargs='*')
parser_unset = self.subparsers.add_parser(
'unset',
help="""Comment out the #define for SYMBOL. Do nothing if none is present.""")
parser_unset.add_argument('symbol', metavar='SYMBOL')
parser_unset_all = self.subparsers.add_parser(
'unset-all',
help="""Comment out all #define whose name contains a match for REGEX.""")
parser_unset_all.add_argument('regexs', metavar='REGEX', nargs='*')
parser_get_all = self.subparsers.add_parser(
'get-all',
help="""Get all #define whose name contains a match for REGEX.""")
parser_get_all.add_argument('regexs', metavar='REGEX', nargs='*')
parser_get_all_enabled = self.subparsers.add_parser(
'get-all-enabled',
help="""Get all enabled #define whose name contains a match for REGEX.""")
parser_get_all_enabled.add_argument('regexs', metavar='REGEX', nargs='*')
def custom_parser_options(self):
"""Adds custom options for the parser. Designed for overridden by descendant."""
pass
def main(self):
# pylint: disable=too-many-branches
"""Common main fuction for config manipulation tool."""
args = self.args
config = self.config
if args.command is None:
self.parser.print_help()
return 1
if args.command == 'get':
if args.symbol in config:
value = config[args.symbol]
if value:
sys.stdout.write(value + '\n')
return 0 if args.symbol in config else 1
elif args.command == 'get-all':
match_list = config.get_matching(args.regexs, False)
sys.stdout.write("\n".join(match_list))
elif args.command == 'get-all-enabled':
match_list = config.get_matching(args.regexs, True)
sys.stdout.write("\n".join(match_list))
elif args.command == 'set':
if not args.force and args.symbol not in config.settings:
sys.stderr.write(
"A #define for the symbol {} was not found in {}\n"
.format(args.symbol,
config.filename(args.symbol)))
return 1
config.set(args.symbol, value=args.value)
elif args.command == 'set-all':
config.change_matching(args.regexs, True)
elif args.command == 'unset':
config.unset(args.symbol)
elif args.command == 'unset-all':
config.change_matching(args.regexs, False)
else:
config.adapt(args.adapter)
config.write(args.write)
return 0

View File

@@ -0,0 +1,59 @@
"""Historical information about the library configuration.
Note: this module is deprecated. Use config_macros.py instead.
"""
## Copyright The Mbed TLS Contributors
## SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import glob
import os
import re
from typing import Dict, FrozenSet
from . import build_tree
class ConfigHistory:
"""Historical information about the library configuration.
"""
@staticmethod
def load_options(filename: str) -> FrozenSet[str]:
"""Load the options from the given file.
The file must contain one option name per line, with no whitespace.
"""
with open(filename) as inp:
return frozenset(inp.read().splitlines())
def load_history_files(self,
variety: str,
exclude_dict: Dict[str, Dict[str, FrozenSet[str]]]
) -> Dict[str, Dict[str, FrozenSet[str]]]:
"""Load all files containing macro lists of the given variety"""
data = {} #type: Dict[str, Dict[str, FrozenSet[str]]]
glob_pattern = os.path.join(self._history_dir,
f'config-{variety}-*-*.txt')
for filename in sorted(glob.glob(glob_pattern)):
basename = os.path.splitext(os.path.basename(filename))[0]
m = re.match(r'config-\w+-(.*?)-(.*)', basename)
assert m is not None
(product, version) = m.groups()
options = self.load_options(filename)
exclude = exclude_dict.get(product, {}).get(version, frozenset())
data.setdefault(product, {})[version] = options - exclude
return data
def __init__(self) -> None:
"""Load all files containing historical option lists."""
self._history_dir = os.path.join(build_tree.framework_root(), 'history')
self._options = self.load_history_files('options', {})
self._internal = self.load_history_files('adjust', self._options)
def options(self, product: str, version: str) -> FrozenSet[str]:
"""The set of options in the given product version."""
return self._options[product][version]
def internal(self, product: str, version: str) -> FrozenSet[str]:
"""The set of internal option-like macros in the given product version."""
return self._internal[product][version]

View File

@@ -0,0 +1,180 @@
"""Information about configuration macros and derived macros."""
## Copyright The Mbed TLS Contributors
## SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import glob
import os
import re
from typing import FrozenSet, Iterable, Iterator, List
from . import build_tree
from . import generate_files_helper
class ConfigMacros:
"""Information about configuration macros and derived macros."""
def __init__(self, public: FrozenSet[str], adjusted: FrozenSet[str]) -> None:
self._public = public
self._internal = adjusted - public
def options(self) -> FrozenSet[str]:
"""The set of configuration options in this product."""
return self._public
def internal(self) -> FrozenSet[str]:
"""The set of internal option-like macros in this product."""
return self._internal
@staticmethod
def _load_file(filename: str) -> FrozenSet[str]:
"""Load macro names from the given file."""
with open(filename, encoding='ascii') as input_:
return frozenset(line.strip()
for line in input_)
class Current(ConfigMacros, generate_files_helper.Generator):
"""Information about config-like macros parsed from the source code."""
_SHADOW_FILE = 'scripts/data_files/config-options-current.txt'
_PUBLIC_CONFIG_HEADERS = [
'include/mbedtls/mbedtls_config.h',
'include/psa/crypto_config.h',
]
_ADJUST_CONFIG_HEADERS = [
'include/**/*adjust*.h',
'drivers/*/include/**/*adjust*.h',
]
_DEFINE_RE = re.compile(r'[/ ]*# *define *([A-Z_a-z][0-9A-Z_a-z]*)')
def _list_files(self, patterns: Iterable[str]) -> Iterator[str]:
"""Yield files matching the given glob patterns."""
for pattern in patterns:
yield from glob.glob(os.path.join(self._root, self._submodule,
pattern),
recursive=True)
def _search_file(self, filename: str) -> Iterator[str]:
"""Yield macros defined in the given file."""
with open(filename, encoding='utf-8') as input_:
for line in input_:
m = self._DEFINE_RE.match(line)
if m:
yield m.group(1)
def _search_files(self, patterns: Iterable[str]) -> FrozenSet[str]:
"""Yield macros defined in files matching the given glob patterns."""
return frozenset(element
for filename in self._list_files(patterns)
for element in self._search_file(filename))
def shadow_file_path(self) -> str:
"""The path to the option list shadow file."""
return os.path.join(self._root, self._submodule, self._SHADOW_FILE)
def __init__(self, submodule: str = '',
shadow_missing_ok: bool = False) -> None:
"""Look for macros defined in the given submodule's source tree.
If submodule is omitted or empty, look in the root module.
If shadow_missing_ok is true, treat a missing shadow file as
if it was empty. This is intended for use only when regenerating
the shadow file.
"""
self._root = build_tree.guess_project_root()
self._submodule = submodule
shadow_file = self.shadow_file_path()
try:
public = self._load_file(shadow_file)
except FileNotFoundError:
if not shadow_missing_ok:
raise
public = frozenset()
adjusted = self._search_files(self._ADJUST_CONFIG_HEADERS)
super().__init__(public, adjusted)
def live_config_options(self) -> FrozenSet[str]:
"""Return config options from the config file (as opposed to the shadow file)."""
return self._search_files(self._PUBLIC_CONFIG_HEADERS)
def is_shadow_file_up_to_date(self) -> bool:
"""Whether the config options shadow file is up to date."""
live = self.live_config_options()
return live == self._public
def compare_shadow_file(self) -> str:
"""Compare the option list shadow file with the live config file.
Return a string containing the names that are only found in one of
them, in a diff-like format: a line prefixed by ``+`` if the name
is missing from the shadow file, or by ``-`` if the name is only
in the shadow file.
"""
live = self.live_config_options()
diff = []
for x in sorted(live | self._public):
if x not in live:
diff.append('+' + x + '\n')
elif x not in self._public:
diff.append('-' + x + '\n')
return ''.join(diff)
def update_shadow_file(self, always_update: bool) -> None:
"""Update the shadow file from the live config file.
If always_update is false and the shadow file already has the desired
content, don't touch it.
"""
if not always_update and self.is_shadow_file_up_to_date():
return
with open(self.shadow_file_path(), 'w') as out:
for name in sorted(self.live_config_options()):
out.write(name + '\n')
# Implement the generate_files_helper.Generator interface
def generator_name(self) -> str:
"""Name as a generate_files_helper.Generator."""
return 'options'
def target_files(self) -> List[str]:
"""List the (single) generated file name."""
return [os.path.join(self._submodule, self._SHADOW_FILE)]
def outdated_files(self) -> List[str]:
"""List the (single) generated file name if it is out of date."""
if self.is_shadow_file_up_to_date():
return []
else:
return self.target_files()
def update(self, always: bool) -> None:
"""Update the shadow file from the live config file."""
self.update_shadow_file(always)
class History(ConfigMacros):
"""Information about config-like macros in a previous version.
Load files created by ``framework/scripts/save_config_history.sh``.
"""
def _load_history_file(self, basename: str) -> FrozenSet[str]:
"""Load macro names from the given file in the history directory."""
filename = os.path.join(self._history_dir, basename)
return self._load_file(filename)
def __init__(self, project: str, version: str) -> None:
"""Read information about the given project at the given version.
The information must be present in history files in the framework.
"""
self._history_dir = os.path.join(build_tree.framework_root(), 'history')
public = self._load_history_file(f'config-options-{project}-{version}.txt')
adjusted = self._load_history_file(f'config-adjust-{project}-{version}.txt')
super().__init__(public, adjusted)

View File

@@ -0,0 +1,106 @@
"""Generate test data for cryptographic mechanisms.
This module is a work in progress, only implementing a few cases for now.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import hashlib
from typing import Callable, Dict, Iterator, List, Optional #pylint: disable=unused-import
from . import crypto_knowledge
from . import psa_information
from . import psa_test_case
from . import test_case
class HashPSALowLevel:
"""Generate test cases for the PSA low-level hash interface."""
def __init__(self, info: psa_information.Information) -> None:
self.info = info
base_algorithms = sorted(info.constructors.algorithms)
all_algorithms = \
[crypto_knowledge.Algorithm(expr)
for expr in info.constructors.generate_expressions(base_algorithms)]
self.algorithms = \
[alg
for alg in all_algorithms
if (not alg.is_wildcard and
alg.can_do(crypto_knowledge.AlgorithmCategory.HASH))]
# CALCULATE[alg] = function to return the hash of its argument in hex
# TO-DO: implement the None entries with a third-party library, because
# hashlib might not have everything, depending on the Python version and
# the underlying OpenSSL. On Ubuntu 16.04, truncated sha512 and sha3/shake
# are not available. On Ubuntu 22.04, md2, md4 and ripemd160 are not
# available. As of Python 3.14, Ascon is not available.
CALCULATE = {
'PSA_ALG_MD5': lambda data: hashlib.md5(data).hexdigest(),
'PSA_ALG_RIPEMD160': None, #lambda data: hashlib.new('ripdemd160').hexdigest()
'PSA_ALG_SHA_1': lambda data: hashlib.sha1(data).hexdigest(),
'PSA_ALG_SHA_224': lambda data: hashlib.sha224(data).hexdigest(),
'PSA_ALG_SHA_256': lambda data: hashlib.sha256(data).hexdigest(),
'PSA_ALG_SHA_256_192': None, #lambda data: hashlib.new('sha256').hexdigest()[:48]
'PSA_ALG_SHA_384': lambda data: hashlib.sha384(data).hexdigest(),
'PSA_ALG_SHA_512': lambda data: hashlib.sha512(data).hexdigest(),
'PSA_ALG_SHA_512_224': None, #lambda data: hashlib.new('sha512_224').hexdigest()
'PSA_ALG_SHA_512_256': None, #lambda data: hashlib.new('sha512_256').hexdigest()
'PSA_ALG_SHA3_224': None, #lambda data: hashlib.sha3_224(data).hexdigest(),
'PSA_ALG_SHA3_256': None, #lambda data: hashlib.sha3_256(data).hexdigest(),
'PSA_ALG_SHA3_384': None, #lambda data: hashlib.sha3_384(data).hexdigest(),
'PSA_ALG_SHA3_512': None, #lambda data: hashlib.sha3_512(data).hexdigest(),
'PSA_ALG_SHAKE128_192': None, #lambda data: hashlib.shake_128(data).hexdigest(24),
'PSA_ALG_SHAKE128_256': None, #lambda data: hashlib.shake_128(data).hexdigest(32),
'PSA_ALG_SHAKE256_256': None, #lambda data: hashlib.shake_256(data).hexdigest(32),
'PSA_ALG_SHAKE256_512': None, #lambda data: hashlib.shake_256(data).hexdigest(64),
'PSA_ALG_ASCON_HASH256': None, #lambda data: ascon.ascon_hash(data),
} #type: Dict[str, Optional[Callable[[bytes], str]]]
@staticmethod
def one_test_case(alg: crypto_knowledge.Algorithm,
function: str, note: str,
arguments: List[str]) -> test_case.TestCase:
"""Construct one test case involving a hash."""
tc = psa_test_case.TestCase(dependency_prefix='MBEDTLS_PSA_BUILTIN_')
tc.set_description('{}{} {}'
.format(function,
' ' + note if note else '',
alg.short_expression()))
tc.set_function(function)
tc.set_arguments([alg.expression] +
['"{}"'.format(arg) for arg in arguments])
return tc
def test_cases_for_hash(self,
alg: crypto_knowledge.Algorithm
) -> Iterator[test_case.TestCase]:
"""Enumerate all test cases for one hash algorithm."""
calc = self.CALCULATE[alg.expression]
if calc is None:
return # not implemented yet
short = b'abc'
hash_short = calc(short)
long = (b'Hello, world. Here are 16 unprintable bytes: ['
b'\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a'
b'\x80\x81\x82\x83\xfe\xff]. '
b' This message was brought to you by a natural intelligence. '
b' If you can read this, good luck with your debugging!')
hash_long = calc(long)
yield self.one_test_case(alg, 'hash_empty', '', [calc(b'')])
yield self.one_test_case(alg, 'hash_valid_one_shot', '',
[short.hex(), hash_short])
for n in [0, 1, 64, len(long) - 1, len(long)]:
yield self.one_test_case(alg, 'hash_valid_multipart',
'{} + {}'.format(n, len(long) - n),
[long[:n].hex(), calc(long[:n]),
long[n:].hex(), hash_long])
def all_test_cases(self) -> Iterator[test_case.TestCase]:
"""Enumerate all test cases for all hash algorithms."""
for alg in self.algorithms:
yield from self.test_cases_for_hash(alg)

View File

@@ -0,0 +1,610 @@
"""Knowledge about cryptographic mechanisms implemented in Mbed TLS.
This module is entirely based on the PSA API.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import enum
import re
from typing import FrozenSet, Iterable, List, Optional, Tuple, Dict
from .asymmetric_key_data import ASYMMETRIC_KEY_DATA
def short_expression(original: str, level: int = 0) -> str:
"""Abbreviate the expression, keeping it human-readable.
If `level` is 0, just remove parts that are implicit from context,
such as a leading ``PSA_KEY_TYPE_``.
For larger values of `level`, also abbreviate some names in an
unambiguous, but ad hoc way.
"""
short = original
short = re.sub(r'\bPSA_(?:ALG|DH_FAMILY|ECC_FAMILY|KEY_[A-Z]+)_', r'', short)
short = re.sub(r' +', r'', short)
if level >= 1:
short = re.sub(r'PUBLIC_KEY\b', r'PUB', short)
short = re.sub(r'KEY_PAIR\b', r'PAIR', short)
short = re.sub(r'\bBRAINPOOL_P', r'BP', short)
short = re.sub(r'\bMONTGOMERY\b', r'MGM', short)
short = re.sub(r'AEAD_WITH_SHORTENED_TAG\b', r'AEAD_SHORT', short)
short = re.sub(r'\bDETERMINISTIC_', r'DET_', short)
short = re.sub(r'\bKEY_AGREEMENT\b', r'KA', short)
short = re.sub(r'_PSK_TO_MS\b', r'_PSK2MS', short)
return short
BLOCK_CIPHERS = frozenset(['AES', 'ARIA', 'CAMELLIA', 'DES'])
BLOCK_MAC_MODES = frozenset(['CBC_MAC', 'CMAC'])
BLOCK_CIPHER_MODES = frozenset([
'CTR', 'CFB', 'OFB', 'XTS', 'CCM_STAR_NO_TAG',
'ECB_NO_PADDING', 'CBC_NO_PADDING', 'CBC_PKCS7',
])
BLOCK_AEAD_MODES = frozenset(['CCM', 'GCM'])
class EllipticCurveCategory(enum.Enum):
"""Categorization of elliptic curve families.
The category of a curve determines what algorithms are defined over it.
"""
SHORT_WEIERSTRASS = 0
MONTGOMERY = 1
TWISTED_EDWARDS = 2
@staticmethod
def from_family(family: str) -> 'EllipticCurveCategory':
if family == 'PSA_ECC_FAMILY_MONTGOMERY':
return EllipticCurveCategory.MONTGOMERY
if family == 'PSA_ECC_FAMILY_TWISTED_EDWARDS':
return EllipticCurveCategory.TWISTED_EDWARDS
# Default to SW, which most curves belong to.
return EllipticCurveCategory.SHORT_WEIERSTRASS
class KeyType:
"""Knowledge about a PSA key type."""
def __init__(self, name: str, params: Optional[Iterable[str]] = None) -> None:
"""Analyze a key type.
The key type must be specified in PSA syntax. In its simplest form,
`name` is a string 'PSA_KEY_TYPE_xxx' which is the name of a PSA key
type macro. For key types that take arguments, the arguments can
be passed either through the optional argument `params` or by
passing an expression of the form 'PSA_KEY_TYPE_xxx(param1, ...)'
in `name` as a string.
"""
self.name = name.strip()
"""The key type macro name (``PSA_KEY_TYPE_xxx``).
For key types constructed from a macro with arguments, this is the
name of the macro, and the arguments are in `self.params`.
"""
if params is None:
if '(' in self.name:
m = re.match(r'(\w+)\s*\((.*)\)\Z', self.name)
assert m is not None
self.name = m.group(1)
params = m.group(2).split(',')
self.params = (None if params is None else
[param.strip() for param in params])
"""The parameters of the key type, if there are any.
None if the key type is a macro without arguments.
"""
assert re.match(r'PSA_KEY_TYPE_\w+\Z', self.name)
self.expression = self.name
"""A C expression whose value is the key type encoding."""
if self.params is not None:
self.expression += '(' + ', '.join(self.params) + ')'
m = re.match(r'PSA_KEY_TYPE_(\w+)', self.name)
assert m
self.head = re.sub(r'_(?:PUBLIC_KEY|KEY_PAIR)\Z', r'', m.group(1))
"""The key type macro name, with common prefixes and suffixes stripped."""
self.private_type = re.sub(r'_PUBLIC_KEY\Z', r'_KEY_PAIR', self.name)
"""The key type macro name for the corresponding key pair type.
For everything other than a public key type, this is the same as
`self.name`.
"""
def short_expression(self, level: int = 0) -> str:
"""Abbreviate the expression, keeping it human-readable.
See `crypto_knowledge.short_expression`.
"""
return short_expression(self.expression, level=level)
def is_public(self) -> bool:
"""Whether the key type is for public keys."""
return self.name.endswith('_PUBLIC_KEY')
DH_KEY_SIZES = {
'PSA_DH_FAMILY_RFC3526': (1536, 2048, 3072, 4096, 6144, 8192),
'PSA_DH_FAMILY_RFC7919': (2048, 3072, 4096, 6144, 8192),
} # type: Dict[str, Tuple[int, ...]]
ECC_KEY_SIZES = {
'PSA_ECC_FAMILY_SECP_K1': (192, 225, 256),
'PSA_ECC_FAMILY_SECP_R1': (224, 256, 384, 521),
'PSA_ECC_FAMILY_SECP_R2': (160,),
'PSA_ECC_FAMILY_SECT_K1': (163, 233, 239, 283, 409, 571),
'PSA_ECC_FAMILY_SECT_R1': (163, 233, 283, 409, 571),
'PSA_ECC_FAMILY_SECT_R2': (163,),
'PSA_ECC_FAMILY_BRAINPOOL_P_R1': (160, 192, 224, 256, 320, 384, 512),
'PSA_ECC_FAMILY_FRP': (256,),
'PSA_ECC_FAMILY_MONTGOMERY': (255, 448),
'PSA_ECC_FAMILY_TWISTED_EDWARDS': (255, 448),
} # type: Dict[str, Tuple[int, ...]]
KEY_TYPE_SIZES = {
'PSA_KEY_TYPE_AES': (128, 192, 256), # exhaustive
'PSA_KEY_TYPE_ARC4': (8, 128, 2048), # extremes + sensible
'PSA_KEY_TYPE_ARIA': (128, 192, 256), # exhaustive
'PSA_KEY_TYPE_ASCON': (128, 256), # exhaustive
'PSA_KEY_TYPE_CAMELLIA': (128, 192, 256), # exhaustive
'PSA_KEY_TYPE_CHACHA20': (256,), # exhaustive
'PSA_KEY_TYPE_DERIVE': (120, 128), # sample
'PSA_KEY_TYPE_DES': (64, 128, 192), # exhaustive
'PSA_KEY_TYPE_HMAC': (128, 160, 224, 256, 384, 512), # standard size for each supported hash
'PSA_KEY_TYPE_PASSWORD': (48, 168, 336), # sample
'PSA_KEY_TYPE_PASSWORD_HASH': (128, 256), # sample
'PSA_KEY_TYPE_PEPPER': (128, 256), # sample
'PSA_KEY_TYPE_RAW_DATA': (8, 40, 128), # sample
'PSA_KEY_TYPE_RSA_KEY_PAIR': (1024, 1536), # small sample
'PSA_KEY_TYPE_SM4': (128,), # exhaustive
'PSA_KEY_TYPE_XCHACHA20': (256,), # exhaustive
} # type: Dict[str, Tuple[int, ...]]
def sizes_to_test(self) -> Tuple[int, ...]:
"""Return a tuple of key sizes to test.
For key types that only allow a single size, or only a small set of
sizes, these are all the possible sizes. For key types that allow a
wide range of sizes, these are a representative sample of sizes,
excluding large sizes for which a typical resource-constrained platform
may run out of memory.
"""
if self.private_type == 'PSA_KEY_TYPE_ECC_KEY_PAIR':
assert self.params is not None
return self.ECC_KEY_SIZES[self.params[0]]
if self.private_type == 'PSA_KEY_TYPE_DH_KEY_PAIR':
assert self.params is not None
return self.DH_KEY_SIZES[self.params[0]]
return self.KEY_TYPE_SIZES[self.private_type]
# "48657265006973206b6579a064617461"
DATA_BLOCK = b'Here\000is key\240data'
def key_material(self, bits: int) -> bytes:
"""Return a byte string containing suitable key material with the given bit length.
Use the PSA export representation. The resulting byte string is one that
can be obtained with the following code:
```
psa_set_key_type(&attributes, `self.expression`);
psa_set_key_bits(&attributes, `bits`);
psa_set_key_usage_flags(&attributes, PSA_KEY_USAGE_EXPORT);
psa_generate_key(&attributes, &id);
psa_export_key(id, `material`, ...);
```
"""
if self.expression in ASYMMETRIC_KEY_DATA:
if bits not in ASYMMETRIC_KEY_DATA[self.expression]:
raise ValueError('No key data for {}-bit {}'
.format(bits, self.expression))
return ASYMMETRIC_KEY_DATA[self.expression][bits]
if bits % 8 != 0:
raise ValueError('Non-integer number of bytes: {} bits for {}'
.format(bits, self.expression))
length = bits // 8
if self.name == 'PSA_KEY_TYPE_DES':
# "644573206b457901644573206b457902644573206b457904"
des3 = b'dEs kEy\001dEs kEy\002dEs kEy\004'
return des3[:length]
return b''.join([self.DATA_BLOCK] * (length // len(self.DATA_BLOCK)) +
[self.DATA_BLOCK[:length % len(self.DATA_BLOCK)]])
def can_do(self, alg: 'Algorithm') -> bool:
"""Whether this key type can be used for operations with the given algorithm.
This function does not currently handle key derivation or PAKE.
"""
#pylint: disable=too-many-branches,too-many-return-statements
if not alg.is_valid_for_operation():
return False
if self.head == 'HMAC' and alg.head == 'HMAC':
return True
if self.head == 'DES':
# 64-bit block ciphers only allow a reduced set of modes.
return alg.head in [
'CBC_NO_PADDING', 'CBC_PKCS7',
'ECB_NO_PADDING',
]
if self.head in BLOCK_CIPHERS and \
alg.head in frozenset.union(BLOCK_MAC_MODES,
BLOCK_CIPHER_MODES,
BLOCK_AEAD_MODES):
if alg.head in ['CMAC', 'OFB'] and \
self.head in ['ARIA', 'CAMELLIA']:
return False # not implemented in Mbed TLS
if alg.head == 'XTS' and self.head != 'AES':
return False # not implemented in Mbed TLS
return True
if self.head == 'CHACHA20' and alg.head == 'CHACHA20_POLY1305':
return True
if self.head in {'ARC4', 'CHACHA20'} and \
alg.head == 'STREAM_CIPHER':
return True
if self.head == 'RSA' and alg.head.startswith('RSA_'):
return True
if alg.category == AlgorithmCategory.KEY_AGREEMENT and \
self.is_public():
# The PSA API does not use public key objects in key agreement
# operations: it imports the public key as a formatted byte string.
# So a public key object with a key agreement algorithm is not
# a valid combination.
return False
if alg.is_invalid_key_agreement_with_derivation():
return False
if self.head == 'ECC':
assert self.params is not None
eccc = EllipticCurveCategory.from_family(self.params[0])
if alg.head == 'ECDH' and \
eccc in {EllipticCurveCategory.SHORT_WEIERSTRASS,
EllipticCurveCategory.MONTGOMERY}:
return True
if alg.head == 'ECDSA' and \
eccc == EllipticCurveCategory.SHORT_WEIERSTRASS:
return True
if alg.head in {'PURE_EDDSA', 'EDDSA_PREHASH'} and \
eccc == EllipticCurveCategory.TWISTED_EDWARDS:
return True
if self.head == 'DH' and alg.head == 'FFDH':
return True
return False
class AlgorithmCategory(enum.Enum):
"""PSA algorithm categories."""
# The numbers are aligned with the category bits in numerical values of
# algorithms. The names match the `PSA_ALG_IS_xxx` macros.
HASH = 2
MAC = 3
CIPHER = 4
AEAD = 5
SIGN = 6
ASYMMETRIC_ENCRYPTION = 7
KEY_DERIVATION = 8
KEY_AGREEMENT = 9
PAKE = 10
KEY_WRAP = 11
KEY_ENCAPSULATION = 12
XOF = 13
def requires_key(self) -> bool:
"""Whether operations in this category are set up with a key."""
return self not in {
self.HASH,
self.XOF,
self.KEY_DERIVATION, # key passed separately from setup
}
def is_asymmetric(self) -> bool:
"""Whether operations in this category involve asymmetric keys."""
return self in {
self.SIGN,
self.ASYMMETRIC_ENCRYPTION,
self.KEY_AGREEMENT,
self.KEY_ENCAPSULATION,
}
class AlgorithmNotRecognized(Exception):
def __init__(self, expr: str) -> None:
super().__init__('Algorithm not recognized: ' + expr)
self.expr = expr
class Algorithm:
"""Knowledge about a PSA algorithm."""
@staticmethod
def determine_base(expr: str) -> str:
"""Return an expression for the "base" of the algorithm.
This strips off variants of algorithms such as MAC truncation.
This function does not attempt to detect invalid inputs.
"""
m = re.match(r'PSA_ALG_(?:'
r'(?:TRUNCATED|AT_LEAST_THIS_LENGTH)_MAC|'
r'AEAD_WITH_(?:SHORTENED|AT_LEAST_THIS_LENGTH)_TAG'
r')\((.*),[^,]+\)\Z', expr)
if m:
expr = m.group(1)
return expr
@staticmethod
def determine_head(expr: str) -> str:
"""Return the head of an algorithm expression.
The head is the first (outermost) constructor, without its PSA_ALG_
prefix, and with some normalization of similar algorithms.
"""
m = re.match(r'PSA_ALG_(?:DETERMINISTIC_)?(\w+)', expr)
if not m:
raise AlgorithmNotRecognized(expr)
head = m.group(1)
if head == 'KEY_AGREEMENT':
m = re.match(r'PSA_ALG_KEY_AGREEMENT\s*\(\s*PSA_ALG_(\w+)', expr)
if not m:
raise AlgorithmNotRecognized(expr)
head = m.group(1)
head = re.sub(r'_ANY\Z', r'', head)
if re.match(r'ED[0-9]+PH\Z', head):
head = 'EDDSA_PREHASH'
return head
CATEGORY_FROM_HEAD = {
'AES_MMO_ZIGBEE': AlgorithmCategory.HASH,
'ASCON_HASH': AlgorithmCategory.HASH,
'SHA': AlgorithmCategory.HASH,
'SHAKE256_512': AlgorithmCategory.HASH,
'MD': AlgorithmCategory.HASH,
'RIPEMD': AlgorithmCategory.HASH,
'SM3': AlgorithmCategory.HASH,
'ANY_HASH': AlgorithmCategory.HASH,
'HMAC': AlgorithmCategory.MAC,
'STREAM_CIPHER': AlgorithmCategory.CIPHER,
'ASCON_AEAD': AlgorithmCategory.AEAD,
'CHACHA20_POLY1305': AlgorithmCategory.AEAD,
'XCHACHA20_POLY1305': AlgorithmCategory.AEAD,
'DSA': AlgorithmCategory.SIGN,
'ECDSA': AlgorithmCategory.SIGN,
'EDDSA': AlgorithmCategory.SIGN,
'HASH_ML_DSA': AlgorithmCategory.SIGN,
'HASH_SLH_DSA': AlgorithmCategory.SIGN,
'HSS': AlgorithmCategory.SIGN,
'LMS': AlgorithmCategory.SIGN,
'ML_DSA': AlgorithmCategory.SIGN,
'PURE_EDDSA': AlgorithmCategory.SIGN,
'SLH_DSA': AlgorithmCategory.SIGN,
'RSA_PSS': AlgorithmCategory.SIGN,
'RSA_PKCS1V15_SIGN': AlgorithmCategory.SIGN,
'XMMS': AlgorithmCategory.SIGN,
'XMMS_MT': AlgorithmCategory.SIGN,
'RSA_PKCS1V15_CRYPT': AlgorithmCategory.ASYMMETRIC_ENCRYPTION,
'RSA_OAEP': AlgorithmCategory.ASYMMETRIC_ENCRYPTION,
'HKDF': AlgorithmCategory.KEY_DERIVATION,
'TLS12_PRF': AlgorithmCategory.KEY_DERIVATION,
'TLS12_PSK_TO_MS': AlgorithmCategory.KEY_DERIVATION,
'TLS12_ECJPAKE_TO_PMS': AlgorithmCategory.KEY_DERIVATION,
'SP800_108': AlgorithmCategory.KEY_DERIVATION,
'PBKDF': AlgorithmCategory.KEY_DERIVATION,
'ECDH': AlgorithmCategory.KEY_AGREEMENT,
'FFDH': AlgorithmCategory.KEY_AGREEMENT,
# KEY_AGREEMENT(...) is a key derivation with a key agreement component
'KEY_AGREEMENT': AlgorithmCategory.KEY_DERIVATION,
'JPAKE': AlgorithmCategory.PAKE,
'SPAKE2P': AlgorithmCategory.PAKE,
'KW': AlgorithmCategory.KEY_WRAP,
'KWP': AlgorithmCategory.KEY_WRAP,
'ECIES_SEC1': AlgorithmCategory.KEY_ENCAPSULATION,
'ML_KEM': AlgorithmCategory.KEY_ENCAPSULATION,
'ASCON_CXOF': AlgorithmCategory.XOF,
'ASCON_XOF': AlgorithmCategory.XOF,
'SHAKE': AlgorithmCategory.XOF,
}
for x in BLOCK_MAC_MODES:
CATEGORY_FROM_HEAD[x] = AlgorithmCategory.MAC
for x in BLOCK_CIPHER_MODES:
CATEGORY_FROM_HEAD[x] = AlgorithmCategory.CIPHER
for x in BLOCK_AEAD_MODES:
CATEGORY_FROM_HEAD[x] = AlgorithmCategory.AEAD
def determine_category(self, expr: str, head: str) -> AlgorithmCategory:
"""Return the category of the given algorithm expression.
This function does not attempt to detect invalid inputs.
"""
prefix = head
while prefix:
if prefix in self.CATEGORY_FROM_HEAD:
return self.CATEGORY_FROM_HEAD[prefix]
if re.match(r'.*[0-9]\Z', prefix):
prefix = re.sub(r'_*[0-9]+\Z', r'', prefix)
else:
prefix = re.sub(r'_*[^_]*\Z', r'', prefix)
raise AlgorithmNotRecognized(expr)
@staticmethod
def determine_wildcard(expr) -> bool:
"""Whether the given algorithm expression is a wildcard.
This function does not attempt to detect invalid inputs.
"""
if re.search(r'\bPSA_ALG_ANY_HASH\b', expr):
return True
if re.search(r'_AT_LEAST_', expr):
return True
return False
def __init__(self, expr: str) -> None:
"""Analyze an algorithm value.
The algorithm must be expressed as a C expression containing only
calls to PSA algorithm constructor macros and numeric literals.
This class is only programmed to handle valid expressions. Invalid
expressions may result in exceptions or in nonsensical results.
"""
self.expression = re.sub(r'\s+', r'', expr)
self.base_expression = self.determine_base(self.expression)
self.head = self.determine_head(self.base_expression)
self.category = self.determine_category(self.base_expression, self.head)
self.is_wildcard = self.determine_wildcard(self.expression)
def get_key_agreement_derivation(self) -> Optional[str]:
"""For a combined key agreement and key derivation algorithm, get the derivation part.
For anything else, return None.
"""
if self.category != AlgorithmCategory.KEY_AGREEMENT:
return None
m = re.match(r'PSA_ALG_KEY_AGREEMENT\(\w+,\s*(.*)\)\Z', self.expression)
if not m:
return None
kdf_alg = m.group(1)
# Assume kdf_alg is either a valid KDF or 0.
if re.match(r'(?:0[Xx])?0+\s*\Z', kdf_alg):
return None
return kdf_alg
KEY_DERIVATIONS_INCOMPATIBLE_WITH_AGREEMENT = frozenset([
'PSA_ALG_TLS12_ECJPAKE_TO_PMS', # secret input in specific format
])
def is_valid_key_agreement_with_derivation(self) -> bool:
"""Whether this is a valid combined key agreement and key derivation algorithm."""
kdf_alg = self.get_key_agreement_derivation()
if kdf_alg is None:
return False
return kdf_alg not in self.KEY_DERIVATIONS_INCOMPATIBLE_WITH_AGREEMENT
def is_invalid_key_agreement_with_derivation(self) -> bool:
"""Whether this is an invalid combined key agreement and key derivation algorithm."""
kdf_alg = self.get_key_agreement_derivation()
if kdf_alg is None:
return False
return kdf_alg in self.KEY_DERIVATIONS_INCOMPATIBLE_WITH_AGREEMENT
def short_expression(self, level: int = 0) -> str:
"""Abbreviate the expression, keeping it human-readable.
See `crypto_knowledge.short_expression`.
"""
return short_expression(self.expression, level=level)
HASH_LENGTH_BYTES = {
'PSA_ALG_AES_MMO_ZIGBEE': 16,
'PSA_ALG_MD2': 16,
'PSA_ALG_MD4': 16,
'PSA_ALG_MD5': 16,
'PSA_ALG_SHA_1': 20,
'PSA_ALG_SM3': 32,
}
HASH_LENGTH_BITS_RE = re.compile(r'([0-9]+)\Z')
@classmethod
def hash_length(cls, alg: str) -> int:
"""The length of the given hash algorithm, in bytes."""
if alg in cls.HASH_LENGTH_BYTES:
return cls.HASH_LENGTH_BYTES[alg]
m = cls.HASH_LENGTH_BITS_RE.search(alg)
if m:
return int(m.group(1)) // 8
raise ValueError('Unknown hash length for ' + alg)
PERMITTED_TAG_LENGTHS = {
'PSA_ALG_CCM': frozenset([4, 6, 8, 10, 12, 14, 16]),
'PSA_ALG_CHACHA20_POLY1305': frozenset([16]),
'PSA_ALG_GCM': frozenset([4, 8, 12, 13, 14, 15, 16]),
}
MAC_LENGTH = {
'PSA_ALG_CBC_MAC': 16, # actually the block cipher length
'PSA_ALG_CMAC': 16, # actually the block cipher length
}
HMAC_RE = re.compile(r'PSA_ALG_HMAC\((.*)\)\Z')
@classmethod
def permitted_truncations(cls, base: str) -> FrozenSet[int]:
"""Permitted output lengths for the given MAC or AEAD base algorithm.
For a MAC algorithm, this is the set of truncation lengths that
Mbed TLS supports.
For an AEAD algorithm, this is the set of truncation lengths that
are permitted by the algorithm specification.
"""
if base in cls.PERMITTED_TAG_LENGTHS:
return cls.PERMITTED_TAG_LENGTHS[base]
max_length = cls.MAC_LENGTH.get(base, None)
if max_length is None:
m = cls.HMAC_RE.match(base)
if m:
max_length = cls.hash_length(m.group(1))
if max_length is None:
raise ValueError('Unknown permitted lengths for ' + base)
return frozenset(range(4, max_length + 1))
TRUNCATED_ALG_RE = re.compile(
r'(?P<face>PSA_ALG_(?:AEAD_WITH_SHORTENED_TAG|TRUNCATED_MAC))'
r'\((?P<base>.*),'
r'(?P<length>0[Xx][0-9A-Fa-f]+|[1-9][0-9]*|0[0-7]*)[LUlu]*\)\Z')
def is_invalid_truncation(self) -> bool:
"""False for a MAC or AEAD algorithm truncated to an invalid length.
True for a MAC or AEAD algorithm truncated to a valid length or to
a length that cannot be determined. True for anything other than
a truncated MAC or AEAD.
"""
m = self.TRUNCATED_ALG_RE.match(self.expression)
if m:
base = m.group('base')
to_length = int(m.group('length'), 0)
permitted_lengths = self.permitted_truncations(base)
if to_length not in permitted_lengths:
return True
return False
def is_valid_for_operation(self) -> bool:
"""Whether this algorithm construction is valid for an operation.
This function assumes that the algorithm is constructed in a
"grammatically" correct way, and only rejects semantically invalid
combinations.
"""
if self.is_wildcard:
return False
if self.is_invalid_truncation():
return False
return True
def can_do(self, category: AlgorithmCategory) -> bool:
"""Whether this algorithm can perform operations in the given category.
"""
if category == self.category:
return True
if category == AlgorithmCategory.KEY_DERIVATION and \
self.is_valid_key_agreement_with_derivation():
return True
return False
def usage_flags(self, public: bool = False) -> List[str]:
"""The list of usage flags describing operations that can perform this algorithm.
If public is true, only return public-key operations, not private-key operations.
"""
if self.category == AlgorithmCategory.HASH:
flags = []
elif self.category == AlgorithmCategory.MAC:
flags = ['SIGN_HASH', 'SIGN_MESSAGE',
'VERIFY_HASH', 'VERIFY_MESSAGE']
elif self.category == AlgorithmCategory.CIPHER or \
self.category == AlgorithmCategory.AEAD:
flags = ['DECRYPT', 'ENCRYPT']
elif self.category == AlgorithmCategory.SIGN:
flags = ['VERIFY_HASH', 'VERIFY_MESSAGE']
if not public:
flags += ['SIGN_HASH', 'SIGN_MESSAGE']
elif self.category == AlgorithmCategory.ASYMMETRIC_ENCRYPTION:
flags = ['ENCRYPT']
if not public:
flags += ['DECRYPT']
elif self.category == AlgorithmCategory.KEY_DERIVATION or \
self.category == AlgorithmCategory.KEY_AGREEMENT:
flags = ['DERIVE']
else:
raise AlgorithmNotRecognized(self.expression)
return ['PSA_KEY_USAGE_' + flag for flag in flags]

View File

@@ -0,0 +1,884 @@
"""Framework classes for generation of ecp test cases."""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
from typing import List
from . import test_data_generation
from . import bignum_common
from . import build_tree
class EcpTarget(test_data_generation.BaseTarget):
#pylint: disable=abstract-method, too-few-public-methods
"""Target for ecp test case generation."""
target_basename = 'test_suite_ecp.generated'
class EcpP192R1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P192 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p192_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP192R1_ENABLED",
"MBEDTLS_ECP_NIST_OPTIM"]
moduli = ["fffffffffffffffffffffffffffffffeffffffffffffffff"] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
"fffffffffffffffffffffffffffffffefffffffffffffffe",
# Modulus + 1
"ffffffffffffffffffffffffffffffff0000000000000000",
# 2^192 - 1
"ffffffffffffffffffffffffffffffffffffffffffffffff",
# Maximum canonical P192 multiplication result
("fffffffffffffffffffffffffffffffdfffffffffffffffc" +
"000000000000000100000000000000040000000000000004"),
# Generate an overflow during reduction
("00000000000000000000000000000001ffffffffffffffff" +
"ffffffffffffffffffffffffffffffff0000000000000000"),
# Generate an overflow during carry reduction
("ffffffffffffffff00000000000000010000000000000000" +
"fffffffffffffffeffffffffffffffff0000000000000000"),
# First 8 number generated by random.getrandbits(384) - seed(2,2)
("cf1822ffbc6887782b491044d5e341245c6e433715ba2bdd" +
"177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("ffed9235288bc781ae66267594c9c9500925e4749b575bd1" +
"3653f8dd9b1f282e4067c3584ee207f8da94e3e8ab73738f"),
("ef8acd128b4f2fc15f3f57ebf30b94fa82523e86feac7eb7" +
"dc38f519b91751dacdbd47d364be8049a372db8f6e405d93"),
("e8624fab5186ee32ee8d7ee9770348a05d300cb90706a045" +
"defc044a09325626e6b58de744ab6cce80877b6f71e1f6d2"),
("2d3d854e061b90303b08c6e33c7295782d6c797f8f7d9b78" +
"2a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"),
("fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f1" +
"5c14bc4a829e07b0829a48d422fe99a22c70501e533c9135"),
("97eeab64ca2ce6bc5d3fd983c34c769fe89204e2e8168561" +
"867e5e15bc01bfce6a27e0dfcbf8754472154e76e4c11ab2"),
("bd143fa9b714210c665d7435c1066932f4767f26294365b2" +
"721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b"),
# Next 2 number generated by random.getrandbits(192)
"47733e847d718d733ff98ff387c56473a7a83ee0761ebfd2",
"cbd4d3e2d4dec9ef83f0be4e80371eb97f81375eecc1cb63"
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
# secp192r1 support has been removed from development, but it's stil
# available in 3.6 branch.
return build_tree.is_mbedtls_3_6()
def arguments(self)-> List[str]:
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP192R1"] + args
class EcpP224R1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P224 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p224_raw"
input_style = "arch_split"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP224R1_ENABLED",
"MBEDTLS_ECP_NIST_OPTIM"]
moduli = ["ffffffffffffffffffffffffffffffff000000000000000000000001"] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
"ffffffffffffffffffffffffffffffff000000000000000000000000",
# Modulus + 1
"ffffffffffffffffffffffffffffffff000000000000000000000002",
# 2^224 - 1
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
# Maximum canonical P224 multiplication result
("fffffffffffffffffffffffffffffffe000000000000000000000000" +
"00000001000000000000000000000000000000000000000000000000"),
# Generate an overflow during reduction
("00000000000000000000000000010000000070000000002000001000" +
"ffffffffffff9fffffffffe00000efff000070000000002000001003"),
# Generate an underflow during reduction
("00000001000000000000000000000000000000000000000000000000" +
"00000000000dc0000000000000000001000000010000000100000003"),
# First 8 number generated by random.getrandbits(448) - seed(2,2)
("da94e3e8ab73738fcf1822ffbc6887782b491044d5e341245c6e4337" +
"15ba2bdd177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("cdbd47d364be8049a372db8f6e405d93ffed9235288bc781ae662675" +
"94c9c9500925e4749b575bd13653f8dd9b1f282e4067c3584ee207f8"),
("defc044a09325626e6b58de744ab6cce80877b6f71e1f6d2ef8acd12" +
"8b4f2fc15f3f57ebf30b94fa82523e86feac7eb7dc38f519b91751da"),
("2d6c797f8f7d9b782a1be9cd8697bbd0e2520e33e44c50556c71c4a6" +
"6148a86fe8624fab5186ee32ee8d7ee9770348a05d300cb90706a045"),
("8f54f8ceacaab39e83844b40ffa9b9f15c14bc4a829e07b0829a48d4" +
"22fe99a22c70501e533c91352d3d854e061b90303b08c6e33c729578"),
("97eeab64ca2ce6bc5d3fd983c34c769fe89204e2e8168561867e5e15" +
"bc01bfce6a27e0dfcbf8754472154e76e4c11ab2fec3f6b32e8d4b8a"),
("a7a83ee0761ebfd2bd143fa9b714210c665d7435c1066932f4767f26" +
"294365b2721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b"),
("74667bffe202849da9643a295a9ac6decbd4d3e2d4dec9ef83f0be4e" +
"80371eb97f81375eecc1cb6347733e847d718d733ff98ff387c56473"),
# Next 2 number generated by random.getrandbits(224)
"eb9ac688b9d39cca91551e8259cc60b17604e4b4e73695c3e652c71a",
"f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f258ebdbfe3"
]
@property
def arg_a(self) -> str:
limbs = 2 * bignum_common.bits_to_limbs(224, self.bits_in_limb)
hex_digits = bignum_common.hex_digits_for_limb(limbs, self.bits_in_limb)
return super().format_arg('{:x}'.format(self.int_a)).zfill(hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
# secp224r1 support has been removed from development, but it's stil
# available in 3.6 branch.
return build_tree.is_mbedtls_3_6()
def arguments(self)-> List[str]:
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP224R1"] + args
class EcpP256R1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P256 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p256_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP256R1_ENABLED",
"MBEDTLS_ECP_NIST_OPTIM"]
moduli = ["ffffffff00000001000000000000000000000000ffffffffffffffffffffffff"] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
"ffffffff00000001000000000000000000000000fffffffffffffffffffffffe",
# Modulus + 1
"ffffffff00000001000000000000000000000001000000000000000000000000",
# 2^256 - 1
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
# Maximum canonical P256 multiplication result
("fffffffe00000002fffffffe0000000100000001fffffffe00000001fffffffc" +
"00000003fffffffcfffffffffffffffffffffffc000000000000000000000004"),
# Generate an overflow during reduction
("0000000000000000000000010000000000000000000000000000000000000000" +
"00000000000000000000000000000000000000000000000000000000ffffffff"),
# Generate an underflow during reduction
("0000000000000000000000000000000000000000000000000000000000000010" +
"ffffffff00000000000000000000000000000000000000000000000000000000"),
# Generate an overflow during carry reduction
("aaaaaaaa00000000000000000000000000000000000000000000000000000000" +
"00000000000000000000000000000000aaaaaaacaaaaaaaaaaaaaaaa00000000"),
# Generate an underflow during carry reduction
("000000000000000000000001ffffffff00000000000000000000000000000000" +
"0000000000000000000000000000000000000002000000020000000100000002"),
# First 8 number generated by random.getrandbits(512) - seed(2,2)
("4067c3584ee207f8da94e3e8ab73738fcf1822ffbc6887782b491044d5e34124" +
"5c6e433715ba2bdd177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("82523e86feac7eb7dc38f519b91751dacdbd47d364be8049a372db8f6e405d93" +
"ffed9235288bc781ae66267594c9c9500925e4749b575bd13653f8dd9b1f282e"),
("e8624fab5186ee32ee8d7ee9770348a05d300cb90706a045defc044a09325626" +
"e6b58de744ab6cce80877b6f71e1f6d2ef8acd128b4f2fc15f3f57ebf30b94fa"),
("829a48d422fe99a22c70501e533c91352d3d854e061b90303b08c6e33c729578" +
"2d6c797f8f7d9b782a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"),
("e89204e2e8168561867e5e15bc01bfce6a27e0dfcbf8754472154e76e4c11ab2" +
"fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f15c14bc4a829e07b0"),
("bd143fa9b714210c665d7435c1066932f4767f26294365b2721dea3bf63f23d0" +
"dbe53fcafb2147df5ca495fa5a91c89b97eeab64ca2ce6bc5d3fd983c34c769f"),
("74667bffe202849da9643a295a9ac6decbd4d3e2d4dec9ef83f0be4e80371eb9" +
"7f81375eecc1cb6347733e847d718d733ff98ff387c56473a7a83ee0761ebfd2"),
("d08f1bb2531d6460f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f25" +
"8ebdbfe3eb9ac688b9d39cca91551e8259cc60b17604e4b4e73695c3e652c71a"),
# Next 2 number generated by random.getrandbits(256)
"c5e2486c44a4a8f69dc8db48e86ec9c6e06f291b2a838af8d5c44a4eb3172062",
"d4c0dca8b4c9e755cc9c3adcf515a8234da4daeb4f3f87777ad1f45ae9500ec9"
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
def arguments(self)-> List[str]:
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP256R1"] + args
class EcpP384R1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P384 fast reduction."""
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p384_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP384R1_ENABLED",
"MBEDTLS_ECP_NIST_OPTIM"]
moduli = [("ffffffffffffffffffffffffffffffffffffffffffffffff" +
"fffffffffffffffeffffffff0000000000000000ffffffff")
] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
("ffffffffffffffffffffffffffffffffffffffffffffffff" +
"fffffffffffffffeffffffff0000000000000000fffffffe"),
# Modulus + 1
("ffffffffffffffffffffffffffffffffffffffffffffffff" +
"fffffffffffffffeffffffff000000000000000100000000"),
# 2^384 - 1
("ffffffffffffffffffffffffffffffffffffffffffffffff" +
"ffffffffffffffffffffffffffffffffffffffffffffffff"),
# Maximum canonical P384 multiplication result
("ffffffffffffffffffffffffffffffffffffffffffffffff" +
"fffffffffffffffdfffffffe0000000000000001fffffffc" +
"000000000000000000000000000000010000000200000000" +
"fffffffe000000020000000400000000fffffffc00000004"),
# Testing with overflow in A(12) + A(21) + A(20);
("497811378624857a2c2af60d70583376545484cfae5c812f" +
"e2999fc1abb51d18b559e8ca3b50aaf263fdf8f24bdfb98f" +
"ffffffff20e65bf9099e4e73a5e8b517cf4fbeb8fd1750fd" +
"ae6d43f2e53f82d5ffffffffffffffffcc6f1e06111c62e0"),
# Testing with underflow in A(13) + A(22) + A(23) - A(12) - A(20);
("dfdd25e96777406b3c04b8c7b406f5fcf287e1e576003a09" +
"2852a6fbe517f2712b68abef41dbd35183a0614fb7222606" +
"ffffffff84396eee542f18a9189d94396c784059c17a9f18" +
"f807214ef32f2f10ffffffff8a77fac20000000000000000"),
# Testing with overflow in A(23) + A(20) + A(19) - A(22);
("783753f8a5afba6c1862eead1deb2fcdd907272be3ffd185" +
"42b24a71ee8b26cab0aa33513610ff973042bbe1637cc9fc" +
"99ad36c7f703514572cf4f5c3044469a8f5be6312c19e5d3" +
"f8fc1ac6ffffffffffffffff8c86252400000000ffffffff"),
# Testing with underflow in A(23) + A(20) + A(19) - A(22);
("65e1d2362fce922663b7fd517586e88842a9b4bd092e93e6" +
"251c9c69f278cbf8285d99ae3b53da5ba36e56701e2b17c2" +
"25f1239556c5f00117fa140218b46ebd8e34f50d0018701f" +
"a8a0a5cc00000000000000004410bcb4ffffffff00000000"),
# Testing the second round of carry reduction
("000000000000000000000000ffffffffffffffffffffffff" +
"ffffffffffffffffffffffffffffffff0000000000000000" +
"0000000000000000ffffffff000000000000000000000001" +
"00000000000000000000000000000000ffffffff00000001"),
# First 8 number generated by random.getrandbits(768) - seed(2,2)
("ffed9235288bc781ae66267594c9c9500925e4749b575bd1" +
"3653f8dd9b1f282e4067c3584ee207f8da94e3e8ab73738f" +
"cf1822ffbc6887782b491044d5e341245c6e433715ba2bdd" +
"177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("e8624fab5186ee32ee8d7ee9770348a05d300cb90706a045" +
"defc044a09325626e6b58de744ab6cce80877b6f71e1f6d2" +
"ef8acd128b4f2fc15f3f57ebf30b94fa82523e86feac7eb7" +
"dc38f519b91751dacdbd47d364be8049a372db8f6e405d93"),
("fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f1" +
"5c14bc4a829e07b0829a48d422fe99a22c70501e533c9135" +
"2d3d854e061b90303b08c6e33c7295782d6c797f8f7d9b78" +
"2a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"),
("bd143fa9b714210c665d7435c1066932f4767f26294365b2" +
"721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b" +
"97eeab64ca2ce6bc5d3fd983c34c769fe89204e2e8168561" +
"867e5e15bc01bfce6a27e0dfcbf8754472154e76e4c11ab2"),
("8ebdbfe3eb9ac688b9d39cca91551e8259cc60b17604e4b4" +
"e73695c3e652c71a74667bffe202849da9643a295a9ac6de" +
"cbd4d3e2d4dec9ef83f0be4e80371eb97f81375eecc1cb63" +
"47733e847d718d733ff98ff387c56473a7a83ee0761ebfd2"),
("d4c0dca8b4c9e755cc9c3adcf515a8234da4daeb4f3f8777" +
"7ad1f45ae9500ec9c5e2486c44a4a8f69dc8db48e86ec9c6" +
"e06f291b2a838af8d5c44a4eb3172062d08f1bb2531d6460" +
"f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f25"),
("0227eeb7b9d7d01f5769da05d205bbfcc8c69069134bccd3" +
"e1cf4f589f8e4ce0af29d115ef24bd625dd961e6830b54fa" +
"7d28f93435339774bb1e386c4fd5079e681b8f5896838b76" +
"9da59b74a6c3181c81e220df848b1df78feb994a81167346"),
("d322a7353ead4efe440e2b4fda9c025a22f1a83185b98f5f" +
"c11e60de1b343f52ea748db9e020307aaeb6db2c3a038a70" +
"9779ac1f45e9dd320c855fdfa7251af0930cdbd30f0ad2a8" +
"1b2d19a2beaa14a7ff3fe32a30ffc4eed0a7bd04e85bfcdd"),
# Next 2 number generated by random.getrandbits(384)
("5c3747465cc36c270e8a35b10828d569c268a20eb78ac332" +
"e5e138e26c4454b90f756132e16dce72f18e859835e1f291"),
("eb2b5693babb7fbb0a76c196067cfdcb11457d9cf45e2fa0" +
"1d7f4275153924800600571fac3a5b263fdf57cd2c006497")
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
def arguments(self)-> List[str]:
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP384R1"] + args
class EcpP521R1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P521 fast reduction."""
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p521_raw"
input_style = "arch_split"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP521R1_ENABLED",
"MBEDTLS_ECP_NIST_OPTIM"]
moduli = [("01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff")
] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
("01ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe"),
# Modulus + 1
("020000000000000000000000000000000000000000000000000000000000000000" +
"000000000000000000000000000000000000000000000000000000000000000000"),
# Maximum canonical P521 multiplication result
("0003ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"fffff800" +
"0000000000000000000000000000000000000000000000000000000000000000" +
"0000000000000000000000000000000000000000000000000000000000000004"),
# Test case for overflow during addition
("0001efffffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"000001ef" +
"0000000000000000000000000000000000000000000000000000000000000000" +
"000000000000000000000000000000000000000000000000000000000f000000"),
# First 8 number generated by random.getrandbits(1042) - seed(2,2)
("0003cc2e82523e86feac7eb7dc38f519b91751dacdbd47d364be8049a372db8f" +
"6e405d93ffed9235288bc781ae66267594c9c9500925e4749b575bd13653f8dd" +
"9b1f282e" +
"4067c3584ee207f8da94e3e8ab73738fcf1822ffbc6887782b491044d5e34124" +
"5c6e433715ba2bdd177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("00017052829e07b0829a48d422fe99a22c70501e533c91352d3d854e061b9030" +
"3b08c6e33c7295782d6c797f8f7d9b782a1be9cd8697bbd0e2520e33e44c5055" +
"6c71c4a6" +
"6148a86fe8624fab5186ee32ee8d7ee9770348a05d300cb90706a045defc044a" +
"09325626e6b58de744ab6cce80877b6f71e1f6d2ef8acd128b4f2fc15f3f57eb"),
("00021f15a7a83ee0761ebfd2bd143fa9b714210c665d7435c1066932f4767f26" +
"294365b2721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b97eeab64" +
"ca2ce6bc" +
"5d3fd983c34c769fe89204e2e8168561867e5e15bc01bfce6a27e0dfcbf87544" +
"72154e76e4c11ab2fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f1"),
("000381bc2a838af8d5c44a4eb3172062d08f1bb2531d6460f0caeef038c89b38" +
"a8acb5137c9260dc74e088a9b9492f258ebdbfe3eb9ac688b9d39cca91551e82" +
"59cc60b1" +
"7604e4b4e73695c3e652c71a74667bffe202849da9643a295a9ac6decbd4d3e2" +
"d4dec9ef83f0be4e80371eb97f81375eecc1cb6347733e847d718d733ff98ff3"),
("00034816c8c69069134bccd3e1cf4f589f8e4ce0af29d115ef24bd625dd961e6" +
"830b54fa7d28f93435339774bb1e386c4fd5079e681b8f5896838b769da59b74" +
"a6c3181c" +
"81e220df848b1df78feb994a81167346d4c0dca8b4c9e755cc9c3adcf515a823" +
"4da4daeb4f3f87777ad1f45ae9500ec9c5e2486c44a4a8f69dc8db48e86ec9c6"),
("000397846c4454b90f756132e16dce72f18e859835e1f291d322a7353ead4efe" +
"440e2b4fda9c025a22f1a83185b98f5fc11e60de1b343f52ea748db9e020307a" +
"aeb6db2c" +
"3a038a709779ac1f45e9dd320c855fdfa7251af0930cdbd30f0ad2a81b2d19a2" +
"beaa14a7ff3fe32a30ffc4eed0a7bd04e85bfcdd0227eeb7b9d7d01f5769da05"),
("00002c3296e6bc4d62b47204007ee4fab105d83e85e951862f0981aebc1b00d9" +
"2838e766ef9b6bf2d037fe2e20b6a8464174e75a5f834da70569c018eb2b5693" +
"babb7fbb" +
"0a76c196067cfdcb11457d9cf45e2fa01d7f4275153924800600571fac3a5b26" +
"3fdf57cd2c0064975c3747465cc36c270e8a35b10828d569c268a20eb78ac332"),
("00009d23b4917fc09f20dbb0dcc93f0e66dfe717c17313394391b6e2e6eacb0f" +
"0bb7be72bd6d25009aeb7fa0c4169b148d2f527e72daf0a54ef25c0707e33868" +
"7d1f7157" +
"5653a45c49390aa51cf5192bbf67da14be11d56ba0b4a2969d8055a9f03f2d71" +
"581d8e830112ff0f0948eccaf8877acf26c377c13f719726fd70bddacb4deeec"),
# Next 2 number generated by random.getrandbits(521)
("12b84ae65e920a63ac1f2b64df6dff07870c9d531ae72a47403063238da1a1fe" +
"3f9d6a179fa50f96cd4aff9261aa92c0e6f17ec940639bc2ccdf572df00790813e3"),
("166049dd332a73fa0b26b75196cf87eb8a09b27ec714307c68c425424a1574f1" +
"eedf5b0f16cdfdb839424d201e653f53d6883ca1c107ca6e706649889c0c7f38608")
]
@property
def arg_a(self) -> str:
# Number of limbs: 2 * N
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
def arguments(self)-> List[str]:
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP521R1"] + args
class EcpP192K1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P192K1 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p192k1_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP192K1_ENABLED"]
moduli = ["fffffffffffffffffffffffffffffffffffffffeffffee37"] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
"fffffffffffffffffffffffffffffffffffffffeffffee36",
# Modulus + 1
"fffffffffffffffffffffffffffffffffffffffeffffee38",
# 2^192 - 1
"ffffffffffffffffffffffffffffffffffffffffffffffff",
# Maximum canonical P192K1 multiplication result
("fffffffffffffffffffffffffffffffffffffffdffffdc6c" +
"0000000000000000000000000000000100002394013c7364"),
# Test case for overflow during addition
("00000007ffff71b809e27dd832cfd5e04d9d2dbb9f8da217" +
"0000000000000000000000000000000000000000520834f0"),
# First 8 number generated by random.getrandbits(384) - seed(2,2)
("cf1822ffbc6887782b491044d5e341245c6e433715ba2bdd" +
"177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("ffed9235288bc781ae66267594c9c9500925e4749b575bd1" +
"3653f8dd9b1f282e4067c3584ee207f8da94e3e8ab73738f"),
("ef8acd128b4f2fc15f3f57ebf30b94fa82523e86feac7eb7" +
"dc38f519b91751dacdbd47d364be8049a372db8f6e405d93"),
("e8624fab5186ee32ee8d7ee9770348a05d300cb90706a045" +
"defc044a09325626e6b58de744ab6cce80877b6f71e1f6d2"),
("2d3d854e061b90303b08c6e33c7295782d6c797f8f7d9b78" +
"2a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"),
("fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f1" +
"5c14bc4a829e07b0829a48d422fe99a22c70501e533c9135"),
("97eeab64ca2ce6bc5d3fd983c34c769fe89204e2e8168561" +
"867e5e15bc01bfce6a27e0dfcbf8754472154e76e4c11ab2"),
("bd143fa9b714210c665d7435c1066932f4767f26294365b2" +
"721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b"),
# Next 2 number generated by random.getrandbits(192)
"47733e847d718d733ff98ff387c56473a7a83ee0761ebfd2",
"cbd4d3e2d4dec9ef83f0be4e80371eb97f81375eecc1cb63"
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
# secp192k1 support has been removed from development, but it's stil
# available in 3.6 branch.
return build_tree.is_mbedtls_3_6()
def arguments(self):
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP192K1"] + args
class EcpP224K1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P224 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p224k1_raw"
input_style = "arch_split"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP224K1_ENABLED"]
moduli = ["fffffffffffffffffffffffffffffffffffffffffffffffeffffe56d"] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
"fffffffffffffffffffffffffffffffffffffffffffffffeffffe56c",
# Modulus + 1
"fffffffffffffffffffffffffffffffffffffffffffffffeffffe56e",
# 2^224 - 1
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
# Maximum canonical P224K1 multiplication result
("fffffffffffffffffffffffffffffffffffffffffffffffdffffcad8" +
"00000000000000000000000000000000000000010000352802c26590"),
# Test case for overflow during addition
("0000007ffff2b68161180fd8cd92e1a109be158a19a99b1809db8032" +
"0000000000000000000000000000000000000000000000000bf04f49"),
# First 8 number generated by random.getrandbits(448) - seed(2,2)
("da94e3e8ab73738fcf1822ffbc6887782b491044d5e341245c6e4337" +
"15ba2bdd177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("cdbd47d364be8049a372db8f6e405d93ffed9235288bc781ae662675" +
"94c9c9500925e4749b575bd13653f8dd9b1f282e4067c3584ee207f8"),
("defc044a09325626e6b58de744ab6cce80877b6f71e1f6d2ef8acd12" +
"8b4f2fc15f3f57ebf30b94fa82523e86feac7eb7dc38f519b91751da"),
("2d6c797f8f7d9b782a1be9cd8697bbd0e2520e33e44c50556c71c4a6" +
"6148a86fe8624fab5186ee32ee8d7ee9770348a05d300cb90706a045"),
("8f54f8ceacaab39e83844b40ffa9b9f15c14bc4a829e07b0829a48d4" +
"22fe99a22c70501e533c91352d3d854e061b90303b08c6e33c729578"),
("97eeab64ca2ce6bc5d3fd983c34c769fe89204e2e8168561867e5e15" +
"bc01bfce6a27e0dfcbf8754472154e76e4c11ab2fec3f6b32e8d4b8a"),
("a7a83ee0761ebfd2bd143fa9b714210c665d7435c1066932f4767f26" +
"294365b2721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b"),
("74667bffe202849da9643a295a9ac6decbd4d3e2d4dec9ef83f0be4e" +
"80371eb97f81375eecc1cb6347733e847d718d733ff98ff387c56473"),
# Next 2 number generated by random.getrandbits(224)
("eb9ac688b9d39cca91551e8259cc60b17604e4b4e73695c3e652c71a"),
("f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f258ebdbfe3"),
]
@property
def arg_a(self) -> str:
limbs = 2 * bignum_common.bits_to_limbs(224, self.bits_in_limb)
hex_digits = bignum_common.hex_digits_for_limb(limbs, self.bits_in_limb)
return super().format_arg('{:x}'.format(self.int_a)).zfill(hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
# secp224k1 support has been removed from development, but it's stil
# available in 3.6 branch.
return build_tree.is_mbedtls_3_6()
def arguments(self):
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP224K1"] + args
class EcpP256K1Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P256 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p256k1_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_SECP256K1_ENABLED"]
moduli = ["fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2f"] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
"fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc2e",
# Modulus + 1
"fffffffffffffffffffffffffffffffffffffffffffffffffffffffefffffc30",
# 2^256 - 1
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
# Maximum canonical P256K1 multiplication result
("fffffffffffffffffffffffffffffffffffffffffffffffffffffffdfffff85c" +
"000000000000000000000000000000000000000000000001000007a4000e9844"),
# Test case for overflow during addition
("0000fffffc2f000e90a0c86a0a63234e5ba641f43a7e4aecc4040e67ec850562" +
"00000000000000000000000000000000000000000000000000000000585674fd"),
# Test case for overflow during addition
("0000fffffc2f000e90a0c86a0a63234e5ba641f43a7e4aecc4040e67ec850562" +
"00000000000000000000000000000000000000000000000000000000585674fd"),
# First 8 number generated by random.getrandbits(512) - seed(2,2)
("4067c3584ee207f8da94e3e8ab73738fcf1822ffbc6887782b491044d5e34124" +
"5c6e433715ba2bdd177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("82523e86feac7eb7dc38f519b91751dacdbd47d364be8049a372db8f6e405d93" +
"ffed9235288bc781ae66267594c9c9500925e4749b575bd13653f8dd9b1f282e"),
("e8624fab5186ee32ee8d7ee9770348a05d300cb90706a045defc044a09325626" +
"e6b58de744ab6cce80877b6f71e1f6d2ef8acd128b4f2fc15f3f57ebf30b94fa"),
("829a48d422fe99a22c70501e533c91352d3d854e061b90303b08c6e33c729578" +
"2d6c797f8f7d9b782a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"),
("e89204e2e8168561867e5e15bc01bfce6a27e0dfcbf8754472154e76e4c11ab2" +
"fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f15c14bc4a829e07b0"),
("bd143fa9b714210c665d7435c1066932f4767f26294365b2721dea3bf63f23d0" +
"dbe53fcafb2147df5ca495fa5a91c89b97eeab64ca2ce6bc5d3fd983c34c769f"),
("74667bffe202849da9643a295a9ac6decbd4d3e2d4dec9ef83f0be4e80371eb9" +
"7f81375eecc1cb6347733e847d718d733ff98ff387c56473a7a83ee0761ebfd2"),
("d08f1bb2531d6460f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f25" +
"8ebdbfe3eb9ac688b9d39cca91551e8259cc60b17604e4b4e73695c3e652c71a"),
# Next 2 number generated by random.getrandbits(256)
("c5e2486c44a4a8f69dc8db48e86ec9c6e06f291b2a838af8d5c44a4eb3172062"),
("d4c0dca8b4c9e755cc9c3adcf515a8234da4daeb4f3f87777ad1f45ae9500ec9"),
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
def arguments(self):
args = super().arguments()
return ["MBEDTLS_ECP_DP_SECP256K1"] + args
class EcpP255Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP 25519 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "mbedtls_ecp_mod_p255_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_CURVE25519_ENABLED"]
moduli = [("7fffffffffffffffffffffffffffffffffffffffffffffffff" +
"ffffffffffffed")] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
("7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffec"),
# Modulus + 1
("7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffee"),
# 2^255 - 1
("7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"),
# Maximum canonical P255 multiplication result
("3fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffec" +
"0000000000000000000000000000000000000000000000000000000000000190"),
# First 8 number generated by random.getrandbits(510) - seed(2,2)
("1019f0d64ee207f8da94e3e8ab73738fcf1822ffbc6887782b491044d5e34124" +
"5c6e433715ba2bdd177219d30e7a269fd95bafc8f2a4d27bdcf4bb99f4bea973"),
("20948fa1feac7eb7dc38f519b91751dacdbd47d364be8049a372db8f6e405d93" +
"ffed9235288bc781ae66267594c9c9500925e4749b575bd13653f8dd9b1f282e"),
("3a1893ea5186ee32ee8d7ee9770348a05d300cb90706a045defc044a09325626" +
"e6b58de744ab6cce80877b6f71e1f6d2ef8acd128b4f2fc15f3f57ebf30b94fa"),
("20a6923522fe99a22c70501e533c91352d3d854e061b90303b08c6e33c729578" +
"2d6c797f8f7d9b782a1be9cd8697bbd0e2520e33e44c50556c71c4a66148a86f"),
("3a248138e8168561867e5e15bc01bfce6a27e0dfcbf8754472154e76e4c11ab2" +
"fec3f6b32e8d4b8a8f54f8ceacaab39e83844b40ffa9b9f15c14bc4a829e07b0"),
("2f450feab714210c665d7435c1066932f4767f26294365b2721dea3bf63f23d0" +
"dbe53fcafb2147df5ca495fa5a91c89b97eeab64ca2ce6bc5d3fd983c34c769f"),
("1d199effe202849da9643a295a9ac6decbd4d3e2d4dec9ef83f0be4e80371eb9" +
"7f81375eecc1cb6347733e847d718d733ff98ff387c56473a7a83ee0761ebfd2"),
("3423c6ec531d6460f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f25" +
"8ebdbfe3eb9ac688b9d39cca91551e8259cc60b17604e4b4e73695c3e652c71a"),
# Next 2 number generated by random.getrandbits(255)
("62f1243644a4a8f69dc8db48e86ec9c6e06f291b2a838af8d5c44a4eb3172062"),
("6a606e54b4c9e755cc9c3adcf515a8234da4daeb4f3f87777ad1f45ae9500ec9"),
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
def arguments(self)-> List[str]:
args = super().arguments()
return ["MBEDTLS_ECP_DP_CURVE25519"] + args
class EcpP448Raw(bignum_common.ModOperationCommon,
EcpTarget):
"""Test cases for ECP P448 fast reduction."""
symbol = "-"
test_function = "ecp_mod_p_generic_raw"
test_name = "ecp_mod_p448_raw"
input_style = "fixed"
arity = 1
dependencies = ["MBEDTLS_ECP_DP_CURVE448_ENABLED"]
moduli = [("fffffffffffffffffffffffffffffffffffffffffffffffffffffffe" +
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff")] # type: List[str]
input_values = [
"0", "1",
# Modulus - 1
("fffffffffffffffffffffffffffffffffffffffffffffffffffffffe" +
"fffffffffffffffffffffffffffffffffffffffffffffffffffffffe"),
# Modulus + 1
("ffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"00000000000000000000000000000000000000000000000000000000"),
# 2^448 - 1
("ffffffffffffffffffffffffffffffffffffffffffffffffffffffff" +
"ffffffffffffffffffffffffffffffffffffffffffffffffffffffff"),
# Maximum canonical P448 multiplication result
("fffffffffffffffffffffffffffffffffffffffffffffffffffffffd" +
"fffffffffffffffffffffffffffffffffffffffffffffffffffffffd" +
"00000000000000000000000000000000000000000000000000000004" +
"00000000000000000000000000000000000000000000000000000004"),
# First 8 number generated by random.getrandbits(896) - seed(2,2)
("74667bffe202849da9643a295a9ac6decbd4d3e2d4dec9ef83f0be4e" +
"80371eb97f81375eecc1cb6347733e847d718d733ff98ff387c56473" +
"a7a83ee0761ebfd2bd143fa9b714210c665d7435c1066932f4767f26" +
"294365b2721dea3bf63f23d0dbe53fcafb2147df5ca495fa5a91c89b"),
("4da4daeb4f3f87777ad1f45ae9500ec9c5e2486c44a4a8f69dc8db48" +
"e86ec9c6e06f291b2a838af8d5c44a4eb3172062d08f1bb2531d6460" +
"f0caeef038c89b38a8acb5137c9260dc74e088a9b9492f258ebdbfe3" +
"eb9ac688b9d39cca91551e8259cc60b17604e4b4e73695c3e652c71a"),
("bc1b00d92838e766ef9b6bf2d037fe2e20b6a8464174e75a5f834da7" +
"0569c018eb2b5693babb7fbb0a76c196067cfdcb11457d9cf45e2fa0" +
"1d7f4275153924800600571fac3a5b263fdf57cd2c0064975c374746" +
"5cc36c270e8a35b10828d569c268a20eb78ac332e5e138e26c4454b9"),
("8d2f527e72daf0a54ef25c0707e338687d1f71575653a45c49390aa5" +
"1cf5192bbf67da14be11d56ba0b4a2969d8055a9f03f2d71581d8e83" +
"0112ff0f0948eccaf8877acf26c377c13f719726fd70bddacb4deeec" +
"0b0c995e96e6bc4d62b47204007ee4fab105d83e85e951862f0981ae"),
("84ae65e920a63ac1f2b64df6dff07870c9d531ae72a47403063238da" +
"1a1fe3f9d6a179fa50f96cd4aff9261aa92c0e6f17ec940639bc2ccd" +
"f572df00790813e32748dd1db4917fc09f20dbb0dcc93f0e66dfe717" +
"c17313394391b6e2e6eacb0f0bb7be72bd6d25009aeb7fa0c4169b14"),
("2bb3b36f29421c4021b7379f0897246a40c270b00e893302aba9e7b8" +
"23fc5ad2f58105748ed5d1b7b310b730049dd332a73fa0b26b75196c" +
"f87eb8a09b27ec714307c68c425424a1574f1eedf5b0f16cdfdb8394" +
"24d201e653f53d6883ca1c107ca6e706649889c0c7f3860895bfa813"),
("af3f5d7841b1256d5c1dc12fb5a1ae519fb8883accda6559caa538a0" +
"9fc9370d3a6b86a7975b54a31497024640332b0612d4050771d7b14e" +
"b6c004cc3b8367dc3f2bb31efe9934ad0809eae3ef232a32b5459d83" +
"fbc46f1aea990e94821d46063b4dbf2ca294523d74115c86188b1044"),
("7430051376e31f5aab63ad02854efa600641b4fa37a47ce41aeffafc" +
"3b45402ac02659fe2e87d4150511baeb198ababb1a16daff3da95cd2" +
"167b75dfb948f82a8317cba01c75f67e290535d868a24b7f627f2855" +
"09167d4126af8090013c3273c02c6b9586b4625b475b51096c4ad652"),
# Corner case which causes maximum overflow
("f4ae65e920a63ac1f2b64df6dff07870c9d531ae72a47403063238da1" +
"a1fe3f9d6a179fa50f96cd4aff9261aa92c0e6f17ec940639bc2ccd0B" +
"519A16DF59C53E0D49B209200F878F362ACE518D5B8BFCF9CDC725E5E" +
"01C06295E8605AF06932B5006D9E556D3F190E8136BF9C643D332"),
# Next 2 number generated by random.getrandbits(448)
("8f54f8ceacaab39e83844b40ffa9b9f15c14bc4a829e07b0829a48d4" +
"22fe99a22c70501e533c91352d3d854e061b90303b08c6e33c729578"),
("97eeab64ca2ce6bc5d3fd983c34c769fe89204e2e8168561867e5e15" +
"bc01bfce6a27e0dfcbf8754472154e76e4c11ab2fec3f6b32e8d4b8a"),
]
@property
def arg_a(self) -> str:
return super().format_arg('{:x}'.format(self.int_a)).zfill(2 * self.hex_digits)
def result(self) -> List[str]:
result = self.int_a % self.int_n
return [self.format_result(result)]
@property
def is_valid(self) -> bool:
return True
def arguments(self):
args = super().arguments()
return ["MBEDTLS_ECP_DP_CURVE448"] + args

View File

@@ -0,0 +1,182 @@
"""Utilities for intermediate files that are generated, but platform-independent
and configuration-independent.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import os
import subprocess
import sys
from typing import Dict, Iterable, List, Sequence, Set
class Generator:
"""An abstract base class for generators of intermediate files."""
def generator_name(self) -> str:
"""A name for this generator.
Generator names must be unique and should not be identical to
the name of any target.
"""
raise NotImplementedError
def target_files(self) -> List[str]:
"""The list of files targeted by this generator.
File names are relative to the project root.
"""
raise NotImplementedError
def outdated_files(self) -> Iterable[str]:
"""Return the list of targets that are out of date.
This is empty after running update().
Missing targets are considered out of date.
"""
raise NotImplementedError
def update(self, always: bool) -> None:
"""Update the target(s) of this generator.
If always is false, avoid changing the output file if it already has
the desired content. If always is true, make sure to update the
time stamp on the output file even if it already has the desired content.
"""
raise NotImplementedError
class TestDataGenerator(Generator):
"""A test data generator script.
Even though the test data generator scripts are written in Python, we
run them as a separate process, because their output depends on the
program name (they write sys.argv[0] in a comment in the .data file).
"""
def __init__(self, script: str) -> None:
"""Run the specified test generator to generate files.
Assume that the script is written in Python and has the command line
interface of test_data_generation.py.
"""
self.script = script
def generator_name(self) -> str:
return os.path.basename(self.script)
def target_files(self) -> List[str]:
output = subprocess.check_output([sys.executable, self.script, '--list'],
encoding='utf-8')
return output.splitlines()
def outdated_files(self) -> List[str]:
output = subprocess.check_output([sys.executable, self.script, '--list-outdated'],
encoding='utf-8')
return output.splitlines()
def update(self, _always) -> None:
subprocess.check_call([sys.executable, self.script])
def assemble(available: Iterable[Generator]) -> Dict[str, Generator]:
"""Assemble the generators into a dictionary with both names and targets as keys."""
by_ident = {} #type: Dict[str, Generator]
for generator in available:
ident = generator.generator_name()
if ident in by_ident:
raise Exception(f'Generator conflict: name "{ident}" of {generator} '
f'already recorded for {by_ident[ident]}')
by_ident[ident] = generator
for ident in generator.target_files():
if ident in by_ident:
raise Exception(f'Generator conflict: target "{ident}" of {generator} '
f'already recorded for {by_ident[ident]}')
by_ident[ident] = generator
return by_ident
def list_names(available: Iterable[Generator]) -> List[str]:
"""Return the list of generator names."""
return sorted(generator.generator_name() for generator in available)
def list_targets(available: Iterable[Generator]) -> List[str]:
"""Return the list of generator targets."""
return sorted(target
for generator in available
for target in generator.target_files())
def select(available: Dict[str, Generator],
wanted: Iterable[str]) -> List[Generator]:
"""Select generators by name or target."""
wanted_names = set() #type: Set[str]
for ident in wanted:
if ident not in available:
raise Exception(f'No generator found for {ident}')
wanted_names.add(ident)
return [available[name] for name in sorted(wanted_names)]
def main(generators: Sequence[Generator],
description: str) -> None:
#pylint: disable=too-many-branches
"""Command line entry point.
"""
parser = argparse.ArgumentParser(description=description)
parser.add_argument('--always-update', '-U',
action='store_true',
help=('Update target files unconditionally '
'(overrides --update)'))
parser.add_argument('--list',
action='store_true',
help='List generator names and targets and exit')
parser.add_argument('--list-names',
action='store_true',
help='List generator names and exit')
parser.add_argument('--list-targets',
action='store_true',
help='List generator targets and exit')
parser.add_argument('--update', '-u',
action='store_true',
help='Update target files if needed')
parser.add_argument('--verbose', '-v',
action='store_true',
help='Be more verbose')
parser.add_argument('idents', nargs='*', metavar='NAME|TARGET',
help='List of generator names or targets (all targets if empty)')
args = parser.parse_args()
if args.list:
args.list_names = True
args.list_targets = True
if args.list_names:
for name in list_names(generators):
print(name)
if args.list_targets:
for target in list_targets(generators):
print(target)
if args.list_names or args.list_targets:
return
if args.idents:
available = assemble(generators)
wanted = select(available, args.idents) #type: Sequence[Generator]
else:
wanted = generators
if args.update or args.always_update:
for generator in wanted:
if args.verbose:
sys.stderr.write(f'Running generator {generator.generator_name()}...\n')
generator.update(args.always_update)
else:
outdated = [] #type: List[str]
for generator in wanted:
if args.verbose:
sys.stderr.write(f'Checking targets of generator {generator.generator_name()}...\n')
outdated += generator.outdated_files()
if outdated:
sys.stderr.write(f'Some targets are missing or out of date.\n')
for target in outdated:
print(target)
sys.stderr.write(f'Run {sys.argv[0]} -u and commit the result.')
sys.exit(1)

View File

@@ -0,0 +1,167 @@
"""Generic code to generate, check and list the generated files
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import filecmp
import shutil
import subprocess
import sys
from pathlib import Path
from typing import List, Optional
class GenerationScript:
"""
Representation of a script generating a configuration independent file.
"""
# pylint: disable=too-few-public-methods,too-many-arguments
def __init__(self, script: Path, files: List[Path],
output_dir_option: Optional[str] = None,
output_file_option: Optional[str] = None,
optional: bool = False) -> None:
# Path from the root of Mbed TLS or TF-PSA-Crypto of the generation script
self.script = script
# Executable to run the script, needed for Windows
if script.suffix == ".py":
self.exe = sys.executable
elif script.suffix == ".pl":
self.exe = "perl"
# List of the default paths from the Mbed TLS or TF-PSA-Crypto root of the
# files the script generates.
self.files = files
# Output directory script argument. Can be an empty string in case it is a
# positional argument.
self.output_dir_option = output_dir_option
# Output file script argument. Can be an empty string in case it is a
# positional argument.
self.output_file_option = output_file_option
# Optional files are skipped in --check mode if they don't exist.
# This normally shouldn't happen, but it can happen during transition
# periods where we're adding a new script or a new file, and a
# consuming repository hasn't been updated yet.
self.optional = optional
def get_generation_script_files(generation_script: str) -> List[Path]:
"""
Get the list of the default paths of the files that a given script
generates. It is assumed that the script supports the "--list" option.
"""
files = []
if generation_script.endswith(".py"):
cmd = [sys.executable]
elif generation_script.endswith(".pl"):
cmd = ["perl"]
cmd += [generation_script, "--list"]
output = subprocess.check_output(cmd, universal_newlines=True)
for line in output.splitlines():
files.append(Path(line))
return files
def get_generated_files(generation_scripts: List[GenerationScript]) -> List[Path]:
"""
List the generated files in Mbed TLS or TF-PSA-Crypto. The path from root
is returned for each generated files.
"""
files = []
for generation_script in generation_scripts:
files += generation_script.files
return files
def make_generated_files(generation_scripts: List[GenerationScript]) -> None:
"""
Generate the configuration independent files in their default location in
the Mbed TLS or TF-PSA-Crypto tree.
"""
for generation_script in generation_scripts:
subprocess.run([generation_script.exe, str(generation_script.script)], check=True)
def check_generated_files(generation_scripts: List[GenerationScript],
root: Path) -> bool:
"""
Check that the given root directory contains the generated files as expected/
generated by this script.
"""
ok = True
for generation_script in generation_scripts:
for file in generation_script.files:
file = root / file
if not file.exists():
# If the script is just being added, allow its files not
# to exist. This can happen, at least, when adding a new
# generation script in crypto: until mbedtls is updated,
# the files from that script won't be present when
# the updated crypto is built from mbedtls development.
if generation_script.optional:
continue
raise Exception(f"Expected generated file does not exist: {file}")
bak_file = file.with_name(file.name + ".bak")
if bak_file.exists():
bak_file.unlink()
file.rename(bak_file)
command = [generation_script.exe, str(generation_script.script)]
if generation_script.output_dir_option is not None:
command += [generation_script.output_dir_option,
str(root / Path(generation_script.files[0].parent))]
elif generation_script.output_file_option is not None:
command += generation_script.output_file_option.split()
command += [str(root / Path(generation_script.files[0]))]
subprocess.run([item for item in command if item.strip()], check=True)
for file in generation_script.files:
file = root / file
bak_file = file.with_name(file.name + ".bak")
if generation_script.optional and not bak_file.exists():
# This file is optional and didn't exist before, so
# there's nothing to compare to, or clean up.
continue
if not filecmp.cmp(file, bak_file):
ok = False
ref_file = file.with_name(file.name + ".ref")
ref_file = root / ref_file
if ref_file.exists():
ref_file.unlink()
shutil.copy(file, ref_file)
print(f"Generated file {file} not identical to the reference one {ref_file}.")
file.unlink()
bak_file.rename(file)
return ok
def main(generation_scripts: List[GenerationScript]) -> int:
"""
Main function of this program
"""
parser = argparse.ArgumentParser()
parser.add_argument('--list', action='store_true',
default=False, help='List generated files.')
parser.add_argument('--root', metavar='DIR',
help='Root of the tree containing the generated files \
to check (default: Mbed TLS or TF-PSA-Cryto root.)')
parser.add_argument('--check', action='store_true',
default=False, help='Check the generated files in root')
args = parser.parse_args()
if args.list:
files = get_generated_files(generation_scripts)
for file in files:
print(str(file))
return 0
elif args.check:
ok = check_generated_files(generation_scripts, Path(args.root or "."))
return 0 if ok else 1
else:
make_generated_files(generation_scripts)
return 0 # Any error causes an exception

View File

@@ -0,0 +1,679 @@
"""This script compares the interfaces of two versions of Mbed TLS, looking
for backward incompatibilities between two different Git revisions within
an Mbed TLS repository. It must be run from the root of a Git working tree.
### How the script works ###
For the source (API) and runtime (ABI) interface compatibility, this script
is a small wrapper around the abi-compliance-checker and abi-dumper tools,
applying them to compare the header and library files.
For the storage format, this script compares the automatically generated
storage tests and the manual read tests, and complains if there is a
reduction in coverage. A change in test data will be signaled as a
coverage reduction since the old test data is no longer present. A change in
how test data is presented will be signaled as well; this would be a false
positive.
The results of the API/ABI comparison are either formatted as HTML and stored
at a configurable location, or are given as a brief list of problems.
Returns 0 on success, 1 on non-compliance, and 2 if there is an error
while running the script.
### How to interpret non-compliance ###
This script has relatively common false positives. In many scenarios, it only
reports a pass if there is a strict textual match between the old version and
the new version, and it reports problems where there is a sufficient semantic
match but not a textual match. This section lists some common false positives.
This is not an exhaustive list: in the end what matters is whether we are
breaking a backward compatibility goal.
**API**: the goal is that if an application works with the old version of the
library, it can be recompiled against the new version and will still work.
This is normally validated by comparing the declarations in `include/*/*.h`.
A failure is a declaration that has disappeared or that now has a different
type.
* It's ok to change or remove macros and functions that are documented as
for internal use only or as experimental.
* It's ok to rename function or macro parameters as long as the semantics
has not changed.
* It's ok to change or remove structure fields that are documented as
private.
* It's ok to add fields to a structure that already had private fields
or was documented as extensible.
**ABI**: the goal is that if an application was built against the old version
of the library, the same binary will work when linked against the new version.
This is normally validated by comparing the symbols exported by `libmbed*.so`.
A failure is a symbol that is no longer exported by the same library or that
now has a different type.
* All ABI changes are acceptable if the library version is bumped
(see `scripts/bump_version.sh`).
* ABI changes that concern functions which are declared only inside the
library directory, and not in `include/*/*.h`, are acceptable only if
the function was only ever used inside the same library (libmbedcrypto,
libmbedx509, libmbedtls). As a counter example, if the old version
of libmbedtls calls mbedtls_foo() from libmbedcrypto, and the new version
of libmbedcrypto no longer has a compatible mbedtls_foo(), this does
require a version bump for libmbedcrypto.
**Storage format**: the goal is to check that persistent keys stored by the
old version can be read by the new version. This is normally validated by
comparing the `*read*` test cases in `test_suite*storage_format*.data`.
A failure is a storage read test case that is no longer present with the same
function name and parameter list.
* It's ok if the same test data is present, but its presentation has changed,
for example if a test function is renamed or has different parameters.
* It's ok if redundant tests are removed.
**Generated test coverage**: the goal is to check that automatically
generated tests have as much coverage as before. This is normally validated
by comparing the test cases that are automatically generated by a script.
A failure is a generated test case that is no longer present with the same
function name and parameter list.
* It's ok if the same test data is present, but its presentation has changed,
for example if a test function is renamed or has different parameters.
* It's ok if redundant tests are removed.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import glob
import os
import re
import sys
import traceback
import shutil
import subprocess
import argparse
import logging
import tempfile
import filecmp
import fnmatch
from types import SimpleNamespace
import xml.etree.ElementTree as ET
from . import build_tree
class AbiChecker:
"""API and ABI checker."""
def __init__(self, old_version, new_version, configuration):
"""Instantiate the API/ABI checker.
old_version: RepoVersion containing details to compare against
new_version: RepoVersion containing details to check
configuration.report_dir: directory for output files
configuration.keep_all_reports: if false, delete old reports
configuration.brief: if true, output shorter report to stdout
configuration.check_abi: if true, compare ABIs
configuration.check_api: if true, compare APIs
configuration.check_storage: if true, compare storage format tests
configuration.skip_file: path to file containing symbols and types to skip
"""
self.repo_path = build_tree.guess_project_root()
self.log = None
self.verbose = configuration.verbose
self._setup_logger()
self.report_dir = os.path.abspath(configuration.report_dir)
self.keep_all_reports = configuration.keep_all_reports
self.can_remove_report_dir = not (os.path.exists(self.report_dir) or
self.keep_all_reports)
self.old_version = old_version
self.new_version = new_version
self.skip_file = configuration.skip_file
self.check_abi = configuration.check_abi
self.check_api = configuration.check_api
if self.check_abi != self.check_api:
raise Exception('Checking API without ABI or vice versa is not supported')
self.check_storage_tests = configuration.check_storage
self.brief = configuration.brief
self.git_command = "git"
self.cmake_command = "cmake"
def _setup_logger(self):
self.log = logging.getLogger()
if self.verbose:
self.log.setLevel(logging.DEBUG)
else:
self.log.setLevel(logging.INFO)
self.log.addHandler(logging.StreamHandler())
@staticmethod
def check_abi_tools_are_installed():
for command in ["abi-dumper", "abi-compliance-checker"]:
if not shutil.which(command):
raise Exception("{} not installed, aborting".format(command))
def _get_clean_worktree_for_git_revision(self, version):
"""Make a separate worktree with version.revision checked out.
Do not modify the current worktree."""
git_worktree_path = tempfile.mkdtemp()
if version.repository:
self.log.debug(
"Checking out git worktree for revision {} from {}".format(
version.revision, version.repository
)
)
fetch_output = subprocess.check_output(
[self.git_command, "fetch",
version.repository, version.revision],
cwd=self.repo_path,
stderr=subprocess.STDOUT
)
self.log.debug(fetch_output.decode("utf-8"))
worktree_rev = "FETCH_HEAD"
else:
self.log.debug("Checking out git worktree for revision {}".format(
version.revision
))
worktree_rev = version.revision
worktree_output = subprocess.check_output(
[self.git_command, "worktree", "add", "--detach",
git_worktree_path, worktree_rev],
cwd=self.repo_path,
stderr=subprocess.STDOUT
)
self.log.debug(worktree_output.decode("utf-8"))
version.commit = subprocess.check_output(
[self.git_command, "rev-parse", "HEAD"],
cwd=git_worktree_path,
stderr=subprocess.STDOUT
).decode("ascii").rstrip()
self.log.debug("Commit is {}".format(version.commit))
return git_worktree_path
def _update_git_submodules(self, git_worktree_path):
"""Recursively checkout all submodules at the revision recorded in their
parent module"""
submodule_output = subprocess.check_output(
[self.git_command, "submodule", "foreach", "--recursive",
f'git worktree add --detach "{git_worktree_path}/$displaypath" HEAD'],
cwd=self.repo_path,
stderr=subprocess.STDOUT
)
self.log.debug(submodule_output.decode("utf-8"))
try:
# Try to update the submodules using local commits
# (Git will sometimes insist on fetching the remote without --no-fetch
# if the submodules are shallow clones)
update_output = subprocess.check_output(
[self.git_command, "submodule", "update", "--init", '--recursive', '--no-fetch'],
cwd=git_worktree_path,
stderr=subprocess.STDOUT
)
except subprocess.CalledProcessError as err:
self.log.debug(err.stdout.decode("utf-8"))
# Checkout with --no-fetch failed, falling back to fetching from origin
update_output = subprocess.check_output(
[self.git_command, "submodule", "update", "--init", '--recursive'],
cwd=git_worktree_path,
stderr=subprocess.STDOUT
)
self.log.debug(update_output.decode("utf-8"))
def _build_shared_libraries(self, git_worktree_path, version):
"""Build the shared libraries in the specified worktree."""
build_dir = os.path.join(git_worktree_path, "build")
os.mkdir(build_dir)
configure_output = subprocess.check_output(
[
self.cmake_command, "..",
"-DCMAKE_BUILD_TYPE=Debug",
"-DENABLE_TESTING=OFF",
"-DENABLE_PROGRAMS=OFF",
"-DUSE_SHARED_MBEDTLS_LIBRARY=ON",
"-DUSE_STATIC_MBEDTLS_LIBRARY=OFF",
"-DUSE_SHARED_TF_PSA_CRYPTO_LIBRARY=ON",
"-DUSE_STATIC_TF_PSA_CRYPTO_LIBRARY=OFF",
],
cwd=build_dir,
stderr=subprocess.STDOUT
)
self.log.debug(configure_output.decode("utf-8"))
build_output = subprocess.check_output(
[self.cmake_command, "--build", build_dir],
stderr=subprocess.STDOUT
)
self.log.debug(build_output.decode("utf-8"))
for root, _dirs, files in os.walk(build_dir):
for file in fnmatch.filter(files, "*.so"):
basename = os.path.splitext(file)[0]
path = os.path.join(root, file)
if basename in version.modules:
if not filecmp.cmp(version.modules[basename], path):
raise Exception(
"The following libraries differ, but have the same name:\n"
f"{version.modules[basename]}\n"
f"{path}"
)
else:
version.modules[basename] = path
@staticmethod
def _pretty_revision(version):
if version.revision == version.commit:
return version.revision
else:
return "{} ({})".format(version.revision, version.commit)
def _get_abi_dumps_from_shared_libraries(self, version):
"""Generate the ABI dumps for the specified git revision.
The shared libraries must have been built and the module paths
present in version.modules."""
for mbed_module, module_path in version.modules.items():
output_path = os.path.join(
self.report_dir, "{}-{}-{}.dump".format(
mbed_module, version.revision, version.version
)
)
abi_dump_command = [
"abi-dumper",
module_path,
"-o", output_path,
"-lver", self._pretty_revision(version),
]
abi_dump_output = subprocess.check_output(
abi_dump_command,
stderr=subprocess.STDOUT
)
self.log.debug(abi_dump_output.decode("utf-8"))
version.abi_dumps[mbed_module] = output_path
@staticmethod
def _normalize_storage_test_case_data(line):
"""Eliminate cosmetic or irrelevant details in storage format test cases."""
line = re.sub(r'\s+', r'', line)
return line
def _read_storage_tests(self,
directory,
filename,
is_generated,
storage_tests):
"""Record storage tests from the given file.
Populate the storage_tests dictionary with test cases read from
filename under directory.
"""
at_paragraph_start = True
description = None
full_path = os.path.join(directory, filename)
with open(full_path) as fd:
for line_number, line in enumerate(fd, 1):
line = line.strip()
if not line:
at_paragraph_start = True
continue
if line.startswith('#'):
continue
if at_paragraph_start:
description = line.strip()
at_paragraph_start = False
continue
if line.startswith('depends_on:'):
continue
# We've reached a test case data line
test_case_data = self._normalize_storage_test_case_data(line)
if not is_generated:
# In manual test data, only look at read tests.
function_name = test_case_data.split(':', 1)[0]
if 'read' not in function_name.split('_'):
continue
metadata = SimpleNamespace(
filename=filename,
line_number=line_number,
description=description
)
storage_tests[test_case_data] = metadata
@staticmethod
def _list_generated_test_data_files(git_worktree_path):
"""List the generated test data files."""
generate_psa_tests = 'framework/scripts/generate_psa_tests.py'
if not os.path.isfile(git_worktree_path + '/' + generate_psa_tests):
# The checked-out revision is from before generate_psa_tests.py
# was moved to the framework submodule. Use the old location.
generate_psa_tests = 'tests/scripts/generate_psa_tests.py'
output = subprocess.check_output(
[generate_psa_tests, '--list'],
cwd=git_worktree_path,
).decode('ascii')
return [line for line in output.split('\n') if line]
def _get_storage_format_tests(self, version, git_worktree_path):
"""Record the storage format tests for the specified git version.
The storage format tests are the test suite data files whose name
contains "storage_format".
The version must be checked out at git_worktree_path.
This function creates or updates the generated data files.
"""
# Existing test data files. This may be missing some automatically
# generated files if they haven't been generated yet.
if os.path.isdir(os.path.join(git_worktree_path, 'tf-psa-crypto',
'tests', 'suites')):
storage_data_files = set(glob.glob(
'tf-psa-crypto/tests/suites/test_suite_*storage_format*.data'
))
else:
storage_data_files = set(glob.glob(
'tests/suites/test_suite_*storage_format*.data'
))
# Discover and (re)generate automatically generated data files.
to_be_generated = set()
for filename in self._list_generated_test_data_files(git_worktree_path):
if 'storage_format' in filename:
storage_data_files.add(filename)
to_be_generated.add(filename)
generate_psa_tests = 'framework/scripts/generate_psa_tests.py'
if not os.path.isfile(git_worktree_path + '/' + generate_psa_tests):
# The checked-out revision is from before generate_psa_tests.py
# was moved to the framework submodule. Use the old location.
generate_psa_tests = 'tests/scripts/generate_psa_tests.py'
subprocess.check_call(
[generate_psa_tests] + sorted(to_be_generated),
cwd=git_worktree_path,
)
for test_file in sorted(storage_data_files):
self._read_storage_tests(git_worktree_path,
test_file,
test_file in to_be_generated,
version.storage_tests)
def _cleanup_worktree(self, git_worktree_path):
"""Remove the specified git worktree."""
shutil.rmtree(git_worktree_path)
submodule_output = subprocess.check_output(
[self.git_command, "submodule", "foreach", "--recursive",
f'git worktree remove "{git_worktree_path}/$displaypath"'],
cwd=self.repo_path,
stderr=subprocess.STDOUT
)
self.log.debug(submodule_output.decode("utf-8"))
worktree_output = subprocess.check_output(
[self.git_command, "worktree", "remove", git_worktree_path],
cwd=self.repo_path,
stderr=subprocess.STDOUT
)
self.log.debug(worktree_output.decode("utf-8"))
def _get_abi_dump_for_ref(self, version):
"""Generate the interface information for the specified git revision."""
git_worktree_path = self._get_clean_worktree_for_git_revision(version)
self._update_git_submodules(git_worktree_path)
if self.check_abi:
self._build_shared_libraries(git_worktree_path, version)
self._get_abi_dumps_from_shared_libraries(version)
if self.check_storage_tests:
self._get_storage_format_tests(version, git_worktree_path)
self._cleanup_worktree(git_worktree_path)
def _remove_children_with_tag(self, parent, tag):
children = parent.getchildren()
for child in children:
if child.tag == tag:
parent.remove(child)
else:
self._remove_children_with_tag(child, tag)
def _remove_extra_detail_from_report(self, report_root):
for tag in ['test_info', 'test_results', 'problem_summary',
'added_symbols', 'affected']:
self._remove_children_with_tag(report_root, tag)
for report in report_root:
for problems in report.getchildren()[:]:
if not problems.getchildren():
report.remove(problems)
def _abi_compliance_command(self, mbed_module, output_path):
"""Build the command to run to analyze the library mbed_module.
The report will be placed in output_path."""
abi_compliance_command = [
"abi-compliance-checker",
"-l", mbed_module,
"-old", self.old_version.abi_dumps[mbed_module],
"-new", self.new_version.abi_dumps[mbed_module],
"-strict",
"-report-path", output_path,
]
if self.skip_file:
abi_compliance_command += ["-skip-symbols", self.skip_file,
"-skip-types", self.skip_file]
if self.brief:
abi_compliance_command += ["-report-format", "xml",
"-stdout"]
return abi_compliance_command
def _is_library_compatible(self, mbed_module, compatibility_report):
"""Test if the library mbed_module has remained compatible.
Append a message regarding compatibility to compatibility_report."""
output_path = os.path.join(
self.report_dir, "{}-{}-{}.html".format(
mbed_module, self.old_version.revision,
self.new_version.revision
)
)
try:
subprocess.check_output(
self._abi_compliance_command(mbed_module, output_path),
stderr=subprocess.STDOUT
)
except subprocess.CalledProcessError as err:
if err.returncode != 1:
raise err
if self.brief:
self.log.info(
"Compatibility issues found for {}".format(mbed_module)
)
report_root = ET.fromstring(err.output.decode("utf-8"))
self._remove_extra_detail_from_report(report_root)
self.log.info(ET.tostring(report_root).decode("utf-8"))
else:
self.can_remove_report_dir = False
compatibility_report.append(
"Compatibility issues found for {}, "
"for details see {}".format(mbed_module, output_path)
)
return False
compatibility_report.append(
"No compatibility issues for {}".format(mbed_module)
)
if not (self.keep_all_reports or self.brief):
os.remove(output_path)
return True
@staticmethod
def _is_storage_format_compatible(old_tests, new_tests,
compatibility_report):
"""Check whether all tests present in old_tests are also in new_tests.
Append a message regarding compatibility to compatibility_report.
"""
missing = frozenset(old_tests.keys()).difference(new_tests.keys())
for test_data in sorted(missing):
metadata = old_tests[test_data]
compatibility_report.append(
'Test case from {} line {} "{}" has disappeared: {}'.format(
metadata.filename, metadata.line_number,
metadata.description, test_data
)
)
compatibility_report.append(
'FAIL: {}/{} storage format test cases have changed or disappeared.'.format(
len(missing), len(old_tests)
) if missing else
'PASS: All {} storage format test cases are preserved.'.format(
len(old_tests)
)
)
compatibility_report.append(
'Info: number of storage format tests cases: {} -> {}.'.format(
len(old_tests), len(new_tests)
)
)
return not missing
def get_abi_compatibility_report(self):
"""Generate a report of the differences between the reference ABI
and the new ABI. ABI dumps from self.old_version and self.new_version
must be available."""
compatibility_report = ["Checking evolution from {} to {}".format(
self._pretty_revision(self.old_version),
self._pretty_revision(self.new_version)
)]
compliance_return_code = 0
if self.check_abi:
shared_modules = list(set(self.old_version.modules.keys()) &
set(self.new_version.modules.keys()))
for mbed_module in shared_modules:
if not self._is_library_compatible(mbed_module,
compatibility_report):
compliance_return_code = 1
if self.check_storage_tests:
if not self._is_storage_format_compatible(
self.old_version.storage_tests,
self.new_version.storage_tests,
compatibility_report):
compliance_return_code = 1
for version in [self.old_version, self.new_version]:
for mbed_module, mbed_module_dump in version.abi_dumps.items():
os.remove(mbed_module_dump)
if self.can_remove_report_dir:
os.rmdir(self.report_dir)
self.log.info("\n".join(compatibility_report))
return compliance_return_code
def check_for_abi_changes(self):
"""Generate a report of ABI differences
between self.old_rev and self.new_rev."""
try:
if self.check_api or self.check_abi:
self.check_abi_tools_are_installed()
self._get_abi_dump_for_ref(self.old_version)
self._get_abi_dump_for_ref(self.new_version)
return self.get_abi_compatibility_report()
except subprocess.CalledProcessError as err:
self.log.error(err.stdout.decode("utf-8"))
raise err
def run_main():
try:
parser = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument(
"-v", "--verbose", action="store_true",
help="set verbosity level",
)
parser.add_argument(
"-r", "--report-dir", type=str, default="reports",
help="directory where reports are stored, default is reports",
)
parser.add_argument(
"-k", "--keep-all-reports", action="store_true",
help="keep all reports, even if there are no compatibility issues",
)
parser.add_argument(
"-o", "--old-rev", type=str, help="revision for old version.",
required=True,
)
parser.add_argument(
"-or", "--old-repo", type=str, help="repository for old version."
)
parser.add_argument(
"-n", "--new-rev", type=str, help="revision for new version",
required=True,
)
parser.add_argument(
"-nr", "--new-repo", type=str, help="repository for new version."
)
parser.add_argument(
"-s", "--skip-file", type=str,
help=("path to file containing symbols and types to skip "
"(typically \"-s identifiers\" after running "
"\"tests/scripts/list-identifiers.sh --internal\")")
)
parser.add_argument(
"--check-abi",
action='store_true', default=True,
help="Perform ABI comparison (default: yes)"
)
parser.add_argument("--no-check-abi", action='store_false', dest='check_abi')
parser.add_argument(
"--check-api",
action='store_true', default=True,
help="Perform API comparison (default: yes)"
)
parser.add_argument("--no-check-api", action='store_false', dest='check_api')
parser.add_argument(
"--check-storage",
action='store_true', default=True,
help="Perform storage tests comparison (default: yes)"
)
parser.add_argument("--no-check-storage", action='store_false', dest='check_storage')
parser.add_argument(
"-b", "--brief", action="store_true",
help="output only the list of issues to stdout, instead of a full report",
)
abi_args = parser.parse_args()
if os.path.isfile(abi_args.report_dir):
parser.error("{} is not a directory".format(abi_args.report_dir))
old_version = SimpleNamespace(
version="old",
repository=abi_args.old_repo,
revision=abi_args.old_rev,
commit=None,
abi_dumps={},
storage_tests={},
modules={}
)
new_version = SimpleNamespace(
version="new",
repository=abi_args.new_repo,
revision=abi_args.new_rev,
commit=None,
abi_dumps={},
storage_tests={},
modules={}
)
configuration = SimpleNamespace(
verbose=abi_args.verbose,
report_dir=abi_args.report_dir,
keep_all_reports=abi_args.keep_all_reports,
brief=abi_args.brief,
check_abi=abi_args.check_abi,
check_api=abi_args.check_api,
check_storage=abi_args.check_storage,
skip_file=abi_args.skip_file
)
abi_check = AbiChecker(old_version, new_version, configuration)
return_code = abi_check.check_for_abi_changes()
sys.exit(return_code)
except Exception: # pylint: disable=broad-except
# Print the backtrace and exit explicitly so as to exit with
# status 2, not 1.
traceback.print_exc()
sys.exit(2)

View File

@@ -0,0 +1,46 @@
"""Auxiliary functions used for logging module.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import logging
import sys
def configure_logger(
logger: logging.Logger,
log_format="[%(levelname)s]: %(message)s",
split_level=logging.WARNING
) -> None:
"""
Configure the logging.Logger instance so that:
- Format is set to any log_format.
Default: "[%(levelname)s]: %(message)s"
- loglevel >= split_level are printed to stderr.
- loglevel < split_level are printed to stdout.
Default: logging.WARNING
"""
class MaxLevelFilter(logging.Filter):
# pylint: disable=too-few-public-methods
def __init__(self, max_level, name=''):
super().__init__(name)
self.max_level = max_level
def filter(self, record: logging.LogRecord) -> bool:
return record.levelno <= self.max_level
log_formatter = logging.Formatter(log_format)
# set loglevel >= split_level to be printed to stderr
stderr_hdlr = logging.StreamHandler(sys.stderr)
stderr_hdlr.setLevel(split_level)
stderr_hdlr.setFormatter(log_formatter)
# set loglevel < split_level to be printed to stdout
stdout_hdlr = logging.StreamHandler(sys.stdout)
stdout_hdlr.addFilter(MaxLevelFilter(split_level - 1))
stdout_hdlr.setFormatter(log_formatter)
logger.addHandler(stderr_hdlr)
logger.addHandler(stdout_hdlr)

View File

@@ -0,0 +1,572 @@
"""Collect macro definitions from header files.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import itertools
import re
from typing import Dict, IO, Iterable, Iterator, List, Optional, Pattern, Set, Tuple, Union
class ReadFileLineException(Exception):
def __init__(self, filename: str, line_number: Union[int, str]) -> None:
message = 'in {} at {}'.format(filename, line_number)
super(ReadFileLineException, self).__init__(message)
self.filename = filename
self.line_number = line_number
class read_file_lines:
# Dear Pylint, conventionally, a context manager class name is lowercase.
# pylint: disable=invalid-name,too-few-public-methods
"""Context manager to read a text file line by line.
```
with read_file_lines(filename) as lines:
for line in lines:
process(line)
```
is equivalent to
```
with open(filename, 'r') as input_file:
for line in input_file:
process(line)
```
except that if process(line) raises an exception, then the read_file_lines
snippet annotates the exception with the file name and line number.
"""
def __init__(self, filename: str) -> None:
self.filename = filename
self.file = None #type: Optional[IO[str]]
self.line_number = 'entry' #type: Union[int, str]
self.generator = None #type: Optional[Iterable[Tuple[int, str]]]
def __enter__(self) -> 'read_file_lines':
self.file = open(self.filename)
self.generator = enumerate(self.file)
return self
def __iter__(self) -> Iterator[str]:
assert self.generator is not None
for line_number, content in self.generator:
self.line_number = line_number
yield content
self.line_number = 'exit'
def __exit__(self, exc_type, exc_value, exc_traceback) -> None:
if self.file is not None:
self.file.close()
if exc_type is not None:
raise ReadFileLineException(self.filename, self.line_number) \
from exc_value
class PSAMacroEnumerator:
"""Information about constructors of various PSA Crypto types.
This includes macro names as well as information about their arguments
when applicable.
This class only provides ways to enumerate expressions that evaluate to
values of the covered types. Derived classes are expected to populate
the set of known constructors of each kind, as well as populate
`self.arguments_for` for arguments that are not of a kind that is
enumerated here.
"""
#pylint: disable=too-many-instance-attributes
def __init__(self) -> None:
"""Set up an empty set of known constructor macros.
"""
self.statuses = set() #type: Set[str]
self.lifetimes = set() #type: Set[str]
self.locations = set() #type: Set[str]
self.persistence_levels = set() #type: Set[str]
self.algorithms = set() #type: Set[str]
self.ecc_curves = set() #type: Set[str]
self.dh_groups = set() #type: Set[str]
self.key_types = set() #type: Set[str]
self.key_usage_flags = set() #type: Set[str]
self.hash_algorithms = set() #type: Set[str]
self.mac_algorithms = set() #type: Set[str]
self.key_agreement_algorithms = set() #type: Set[str]
self.key_derivation_algorithms = set() #type: Set[str]
self.pake_algorithms = set() #type: Set[str]
self.aead_algorithms = set() #type: Set[str]
self.sign_algorithms = set() #type: Set[str]
# macro name -> list of argument names
self.argspecs = {} #type: Dict[str, List[str]]
# argument name -> list of values
self.arguments_for = {
'mac_length': [],
'min_mac_length': [],
'tag_length': [],
'min_tag_length': [],
} #type: Dict[str, List[str]]
# Whether to include intermediate macros in enumerations. Intermediate
# macros serve as category headers and are not valid values of their
# type. See `is_internal_name`.
# Always false in this class, may be set to true in derived classes.
self.include_intermediate = False
# Deprecated backward compatibility alias for generate_psa_constants.py.
self.ka_algorithms = self.key_agreement_algorithms
def is_internal_name(self, name: str) -> bool:
"""Whether this is an internal macro. Internal macros will be skipped."""
if not self.include_intermediate:
if name.endswith('_BASE') or name.endswith('_NONE'):
return True
if '_CATEGORY_' in name:
return True
return name.endswith('_FLAG') or name.endswith('_MASK')
def gather_arguments(self) -> None:
"""Populate the list of values for macro arguments.
Call this after parsing all the inputs.
"""
self.arguments_for['hash_alg'] = sorted(self.hash_algorithms)
self.arguments_for['mac_alg'] = sorted(self.mac_algorithms)
self.arguments_for['ka_alg'] = sorted(self.key_agreement_algorithms)
self.arguments_for['kdf_alg'] = sorted(self.key_derivation_algorithms)
self.arguments_for['aead_alg'] = sorted(self.aead_algorithms)
self.arguments_for['sign_alg'] = sorted(self.sign_algorithms)
self.arguments_for['curve'] = sorted(self.ecc_curves)
self.arguments_for['group'] = sorted(self.dh_groups)
self.arguments_for['persistence'] = sorted(self.persistence_levels)
self.arguments_for['location'] = sorted(self.locations)
self.arguments_for['lifetime'] = sorted(self.lifetimes)
@staticmethod
def _format_arguments(name: str, arguments: Iterable[str]) -> str:
"""Format a macro call with arguments.
The resulting format is consistent with
`InputsForTest.normalize_argument`.
"""
return name + '(' + ', '.join(arguments) + ')'
_argument_split_re = re.compile(r' *, *')
@classmethod
def _argument_split(cls, arguments: str) -> List[str]:
return re.split(cls._argument_split_re, arguments)
def distribute_arguments(self, name: str) -> Iterator[str]:
"""Generate macro calls with each tested argument set.
If name is a macro without arguments, just yield "name".
If name is a macro with arguments, yield a series of
"name(arg1,...,argN)" where each argument takes each possible
value at least once.
"""
try:
if name not in self.argspecs:
yield name
return
argspec = self.argspecs[name]
if argspec == []:
yield name + '()'
return
argument_lists = [self.arguments_for[arg] for arg in argspec]
arguments = [values[0] for values in argument_lists]
yield self._format_arguments(name, arguments)
# Dear Pylint, enumerate won't work here since we're modifying
# the array.
# pylint: disable=consider-using-enumerate
for i in range(len(arguments)):
for value in argument_lists[i][1:]:
arguments[i] = value
yield self._format_arguments(name, arguments)
arguments[i] = argument_lists[i][0]
except BaseException as e:
raise Exception('distribute_arguments({})'.format(name)) from e
def distribute_arguments_without_duplicates(
self, seen: Set[str], name: str
) -> Iterator[str]:
"""Same as `distribute_arguments`, but don't repeat seen results."""
for result in self.distribute_arguments(name):
if result not in seen:
seen.add(result)
yield result
def generate_expressions(self, names: Iterable[str]) -> Iterator[str]:
"""Generate expressions covering values constructed from the given names.
`names` can be any iterable collection of macro names.
For example:
* ``generate_expressions(['PSA_ALG_CMAC', 'PSA_ALG_HMAC'])``
generates ``'PSA_ALG_CMAC'`` as well as ``'PSA_ALG_HMAC(h)'`` for
every known hash algorithm ``h``.
* ``macros.generate_expressions(macros.key_types)`` generates all
key types.
"""
seen = set() #type: Set[str]
return itertools.chain(*(
self.distribute_arguments_without_duplicates(seen, name)
for name in names
))
class PSAMacroCollector(PSAMacroEnumerator):
"""Collect PSA crypto macro definitions from C header files.
"""
def __init__(self, include_intermediate: bool = False) -> None:
"""Set up an object to collect PSA macro definitions.
Call the read_file method of the constructed object on each header file.
* include_intermediate: if true, include intermediate macros such as
PSA_XXX_BASE that do not designate semantic values.
"""
super().__init__()
self.include_intermediate = include_intermediate
self.key_types_from_curve = {} #type: Dict[str, str]
self.key_types_from_group = {} #type: Dict[str, str]
self.algorithms_from_hash = {} #type: Dict[str, str]
@staticmethod
def algorithm_tester(name: str) -> str:
"""The predicate for whether an algorithm is built from the given constructor.
The given name must be the name of an algorithm constructor of the
form ``PSA_ALG_xxx`` which is used as ``PSA_ALG_xxx(yyy)`` to build
an algorithm value. Return the corresponding predicate macro which
is used as ``predicate(alg)`` to test whether ``alg`` can be built
as ``PSA_ALG_xxx(yyy)``. The predicate is usually called
``PSA_ALG_IS_xxx``.
"""
prefix = 'PSA_ALG_'
assert name.startswith(prefix)
midfix = 'IS_'
suffix = name[len(prefix):]
if suffix in ['DSA', 'ECDSA']:
midfix += 'RANDOMIZED_'
elif suffix == 'RSA_PSS':
suffix += '_STANDARD_SALT'
return prefix + midfix + suffix
def record_algorithm_subtype(self, name: str, expansion: str) -> None:
"""Record the subtype of an algorithm constructor.
Given a ``PSA_ALG_xxx`` macro name and its expansion, if the algorithm
is of a subtype that is tracked in its own set, add it to the relevant
set.
"""
# This code is very ad hoc and fragile. It should be replaced by
# something more robust.
if re.match(r'MAC(?:_|\Z)', name):
self.mac_algorithms.add(name)
elif re.match(r'KDF(?:_|\Z)', name):
self.key_derivation_algorithms.add(name)
elif re.search(r'0x020000[0-9A-Fa-f]{2}', expansion):
self.hash_algorithms.add(name)
elif re.search(r'0x03[0-9A-Fa-f]{6}', expansion):
self.mac_algorithms.add(name)
elif re.search(r'0x05[0-9A-Fa-f]{6}', expansion):
self.aead_algorithms.add(name)
elif re.search(r'0x09[0-9A-Fa-f]{2}0000', expansion):
self.key_agreement_algorithms.add(name)
elif re.search(r'0x08[0-9A-Fa-f]{6}', expansion):
self.key_derivation_algorithms.add(name)
# "#define" followed by a macro name with either no parameters
# or a single parameter and a non-empty expansion.
# Grab the macro name in group 1, the parameter name if any in group 2
# and the expansion in group 3.
_define_directive_re = re.compile(r'\s*#\s*define\s+(\w+)' +
r'(?:\s+|\((\w+)\)\s*)' +
r'(.+)')
_deprecated_definition_re = re.compile(r'\s*MBEDTLS_DEPRECATED')
# Macro that is a destructor, not a constructor (i.e. takes a thing as
# an argument and analyzes it, rather than constructing a thing).
_destructor_name_re = re.compile('|'.join([
r'.*(?:_GET_|_HAS_|_IS_)',
r'.*_LENGTH\Z',
r'PSA_ALG_SIGN_SUPPORTS_CONTEXT\Z',
]))
# Macro that converts between things, rather than building a thing from
# scratch.
_conversion_macro_names = frozenset([
'PSA_KEY_TYPE_KEY_PAIR_OF_PUBLIC_KEY',
'PSA_KEY_TYPE_PUBLIC_KEY_OF_KEY_PAIR',
'PSA_ALG_FULL_LENGTH_MAC',
'PSA_ALG_AEAD_WITH_DEFAULT_LENGTH_TAG',
'PSA_JPAKE_EXPECTED_INPUTS',
'PSA_JPAKE_EXPECTED_OUTPUTS',
])
def read_line(self, line):
"""Parse a C header line and record the PSA identifier it defines if any.
This function analyzes lines that start with "#define PSA_"
(up to non-significant whitespace) and skips all non-matching lines.
"""
# pylint: disable=too-many-branches,too-many-return-statements
m = re.match(self._define_directive_re, line)
if not m:
return
name, parameter, expansion = m.groups()
expansion = re.sub(r'/\*.*?\*/|//.*', r' ', expansion)
if parameter:
self.argspecs[name] = [parameter]
if re.match(self._deprecated_definition_re, expansion):
# Skip deprecated values, which are assumed to be
# backward compatibility aliases that share
# numerical values with non-deprecated values.
return
if re.match(self._destructor_name_re, name):
# Not a constructor
return
if name in self._conversion_macro_names:
# Not a constructor
return
if self.is_internal_name(name):
# Macro only to build actual values
return
elif (name.startswith('PSA_ERROR_') or name == 'PSA_SUCCESS') \
and not parameter:
self.statuses.add(name)
elif name.startswith('PSA_KEY_TYPE_') and not parameter:
self.key_types.add(name)
elif name.startswith('PSA_KEY_TYPE_') and parameter == 'curve':
self.key_types_from_curve[name] = name[:13] + 'IS_' + name[13:]
elif name.startswith('PSA_KEY_TYPE_') and parameter == 'group':
self.key_types_from_group[name] = name[:13] + 'IS_' + name[13:]
elif name.startswith('PSA_ECC_FAMILY_') and not parameter:
self.ecc_curves.add(name)
elif name.startswith('PSA_DH_FAMILY_') and not parameter:
self.dh_groups.add(name)
elif name.startswith('PSA_ALG_') and not parameter:
if name in ['PSA_ALG_ECDSA_BASE',
'PSA_ALG_RSA_PKCS1V15_SIGN_BASE']:
# Ad hoc skipping of duplicate names for some numerical values
return
self.algorithms.add(name)
self.record_algorithm_subtype(name, expansion)
elif name.startswith('PSA_ALG_') and parameter == 'hash_alg':
self.algorithms_from_hash[name] = self.algorithm_tester(name)
elif name.startswith('PSA_KEY_USAGE_') and not parameter:
self.key_usage_flags.add(name)
elif parameter is None:
# Macro with no parameter, whose name does not start with one
# of the prefixes we look for. Just ignore it.
return
else:
raise Exception("Unsupported macro and parameter name: {}({})"
.format(name, parameter))
_nonascii_re = re.compile(rb'[^\x00-\x7f]+')
_continued_line_re = re.compile(rb'\\\r?\n\Z')
def read_file(self, header_file):
for line in header_file:
m = re.search(self._continued_line_re, line)
while m:
cont = next(header_file)
line = line[:m.start(0)] + cont
m = re.search(self._continued_line_re, line)
line = re.sub(self._nonascii_re, rb'', line).decode('ascii')
self.read_line(line)
class InputsForTest(PSAMacroEnumerator):
# pylint: disable=too-many-instance-attributes
"""Accumulate information about macros to test.
enumerate
This includes macro names as well as information about their arguments
when applicable.
"""
def __init__(self) -> None:
super().__init__()
self.all_declared = set() #type: Set[str]
# Identifier prefixes
self.table_by_prefix = {
'ERROR': self.statuses,
'ALG': self.algorithms,
'ECC_CURVE': self.ecc_curves,
'DH_GROUP': self.dh_groups,
'KEY_LIFETIME': self.lifetimes,
'KEY_LOCATION': self.locations,
'KEY_PERSISTENCE': self.persistence_levels,
'KEY_TYPE': self.key_types,
'KEY_USAGE': self.key_usage_flags,
} #type: Dict[str, Set[str]]
# Test functions
self.table_by_test_function = {
# Any function ending in _algorithm also gets added to
# self.algorithms.
'key_type': [self.key_types],
'block_cipher_key_type': [self.key_types],
'stream_cipher_key_type': [self.key_types],
'ecc_key_family': [self.ecc_curves],
'ecc_key_types': [self.ecc_curves],
'dh_key_family': [self.dh_groups],
'dh_key_types': [self.dh_groups],
'hash_algorithm': [self.hash_algorithms],
'mac_algorithm': [self.mac_algorithms],
'cipher_algorithm': [],
'hmac_algorithm': [self.mac_algorithms, self.sign_algorithms],
'aead_algorithm': [self.aead_algorithms],
'key_derivation_algorithm': [self.key_derivation_algorithms],
'key_agreement_algorithm': [self.key_agreement_algorithms],
'asymmetric_signature_algorithm': [self.sign_algorithms],
'asymmetric_signature_wildcard': [self.algorithms],
'asymmetric_encryption_algorithm': [],
'pake_algorithm': [self.pake_algorithms],
'key_wrap_algorithm': [],
'key_encapsulation_algorithm': [],
'xof_algorithm': [],
'other_algorithm': [],
'lifetime': [self.lifetimes],
} #type: Dict[str, List[Set[str]]]
mac_lengths = [str(n) for n in [
1, # minimum expressible
4, # minimum allowed by policy
13, # an odd size in a plausible range
14, # an even non-power-of-two size in a plausible range
16, # same as full size for at least one algorithm
63, # maximum expressible
]]
self.arguments_for['mac_length'] += mac_lengths
self.arguments_for['min_mac_length'] += mac_lengths
aead_lengths = [str(n) for n in [
1, # minimum expressible
4, # minimum allowed by policy
13, # an odd size in a plausible range
14, # an even non-power-of-two size in a plausible range
16, # same as full size for at least one algorithm
63, # maximum expressible
]]
self.arguments_for['tag_length'] += aead_lengths
self.arguments_for['min_tag_length'] += aead_lengths
def add_numerical_values(self) -> None:
"""Add numerical values that are not supported to the known identifiers."""
# Sets of names per type
self.algorithms.add('0xffffffff')
self.ecc_curves.add('0xff')
self.dh_groups.add('0xff')
self.key_types.add('0xffff')
self.key_usage_flags.add('0x80000000')
# Hard-coded values for unknown algorithms
#
# These have to have values that are correct for their respective
# PSA_ALG_IS_xxx macros, but are also not currently assigned and are
# not likely to be assigned in the near future.
self.hash_algorithms.add('0x020000fe') # 0x020000ff is PSA_ALG_ANY_HASH
self.mac_algorithms.add('0x03007fff')
self.key_agreement_algorithms.add('0x09fc0000')
self.key_derivation_algorithms.add('0x080000ff')
self.pake_algorithms.add('0x0a0000ff')
# For AEAD algorithms, the only variability is over the tag length,
# and this only applies to known algorithms, so don't test an
# unknown algorithm.
def get_names(self, type_word: str) -> Set[str]:
"""Return the set of known names of values of the given type."""
return {
'status': self.statuses,
'algorithm': self.algorithms,
'ecc_curve': self.ecc_curves,
'dh_group': self.dh_groups,
'key_type': self.key_types,
'key_usage': self.key_usage_flags,
}[type_word]
# Regex for interesting header lines.
# Groups: 1=macro name, 2=type, 3=argument list (optional).
_header_line_re = \
re.compile(r'#define +' +
r'(PSA_((?:(?:DH|ECC|KEY)_)?[A-Z]+)_\w+)' +
r'(?:\(([^\n()]*)\))?')
# Regex of macro names to exclude.
_excluded_name_re = re.compile(r'_(?:GET|HAS|IS|OF)_|_(?:BASE|FLAG|MASK)\Z')
# Additional excluded macros.
_excluded_names = set([
# Macros that provide an alternative way to build the same
# algorithm as another macro.
'PSA_ALG_AEAD_WITH_DEFAULT_LENGTH_TAG',
'PSA_ALG_FULL_LENGTH_MAC',
# Auxiliary macro whose name doesn't fit the usual patterns for
# auxiliary macros.
'PSA_ALG_AEAD_WITH_DEFAULT_LENGTH_TAG_CASE',
])
def parse_header_line(self, line: str) -> None:
"""Parse a C header line, looking for "#define PSA_xxx"."""
m = re.match(self._header_line_re, line)
if not m:
return
name = m.group(1)
self.all_declared.add(name)
if re.search(self._excluded_name_re, name) or \
name in self._excluded_names or \
self.is_internal_name(name):
return
dest = self.table_by_prefix.get(m.group(2))
if dest is None:
return
dest.add(name)
if m.group(3):
self.argspecs[name] = self._argument_split(m.group(3))
_nonascii_re = re.compile(rb'[^\x00-\x7f]+') #type: Pattern
def parse_header(self, filename: str) -> None:
"""Parse a C header file, looking for "#define PSA_xxx"."""
with open(filename, 'rb') as input_:
for line in input_:
line = re.sub(self._nonascii_re, rb'', line)
self.parse_header_line(line.decode('ascii'))
_macro_identifier_re = re.compile(r'[A-Z]\w+')
def generate_undeclared_names(self, expr: str) -> Iterable[str]:
for name in re.findall(self._macro_identifier_re, expr):
if name not in self.all_declared:
yield name
def accept_test_case_line(self, function: str, argument: str) -> bool:
#pylint: disable=unused-argument
undeclared = list(self.generate_undeclared_names(argument))
if undeclared:
raise Exception('Undeclared names in test case', undeclared)
return True
@staticmethod
def normalize_argument(argument: str) -> str:
"""Normalize whitespace in the given C expression.
The result uses the same whitespace as
` PSAMacroEnumerator.distribute_arguments`.
"""
return re.sub(r',', r', ', re.sub(r' +', r'', argument))
def add_test_case_line(self, function: str, argument: str) -> None:
"""Parse a test case data line, looking for algorithm metadata tests."""
sets = []
if function.endswith('_algorithm'):
sets.append(self.algorithms)
if function == 'key_agreement_algorithm' and \
argument.startswith('PSA_ALG_KEY_AGREEMENT('):
# We only want *raw* key agreement algorithms as such, so
# exclude ones that are already chained with a KDF.
# Keep the expression as one to test as an algorithm.
function = 'other_algorithm'
sets += self.table_by_test_function[function]
if self.accept_test_case_line(function, argument):
for s in sets:
s.add(self.normalize_argument(argument))
# Regex matching a *.data line containing a test function call and
# its arguments. The actual definition is partly positional, but this
# regex is good enough in practice.
_test_case_line_re = re.compile(r'(?!depends_on:)(\w+):([^\n :][^:\n]*)')
def parse_test_cases(self, filename: str) -> None:
"""Parse a test case file (*.data), looking for algorithm metadata tests."""
with read_file_lines(filename) as lines:
for line in lines:
m = re.match(self._test_case_line_re, line)
if m:
self.add_test_case_line(m.group(1), m.group(2))

View File

@@ -0,0 +1,122 @@
"""Install all the required Python packages, with the minimum Python version.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import os
import re
import subprocess
import sys
import tempfile
import typing
from typing import List, Optional
import framework_scripts_path # pylint: disable=unused-import
from mbedtls_framework import typing_util
def pylint_doesn_t_notice_that_certain_types_are_used_in_annotations(
_list: List[typing.Any],
) -> None:
pass
class Requirements:
"""Collect and massage Python requirements."""
def __init__(self) -> None:
self.requirements = [] #type: List[str]
def adjust_requirement(self, req: str) -> str:
"""Adjust a requirement to the minimum specified version."""
# allow inheritance #pylint: disable=no-self-use
# If a requirement specifies a minimum version, impose that version.
split_req = req.split(';', 1)
split_req[0] = re.sub(r'>=|~=', r'==', split_req[0])
return ';'.join(split_req)
def add_file(self, filename: str) -> None:
"""Add requirements from the specified file.
This method supports a subset of pip's requirement file syntax:
* One requirement specifier per line, which is passed to
`adjust_requirement`.
* Comments (``#`` at the beginning of the line or after whitespace).
* ``-r FILENAME`` to include another file.
"""
for line in open(filename):
line = line.strip()
line = re.sub(r'(\A|\s+)#.*', r'', line)
if not line:
continue
m = re.match(r'-r\s+', line)
if m:
nested_file = os.path.join(os.path.dirname(filename),
line[m.end(0):])
self.add_file(nested_file)
continue
self.requirements.append(self.adjust_requirement(line))
def write(self, out: typing_util.Writable) -> None:
"""List the gathered requirements."""
for req in self.requirements:
out.write(req + '\n')
def install(
self,
pip_general_options: Optional[List[str]] = None,
pip_install_options: Optional[List[str]] = None,
) -> None:
"""Call pip to install the requirements."""
if pip_general_options is None:
pip_general_options = []
if pip_install_options is None:
pip_install_options = []
with tempfile.TemporaryDirectory() as temp_dir:
# This is more complicated than it needs to be for the sake
# of Windows. Use a temporary file rather than the command line
# to avoid quoting issues. Use a temporary directory rather
# than NamedTemporaryFile because with a NamedTemporaryFile on
# Windows, the subprocess can't open the file because this process
# has an exclusive lock on it.
req_file_name = os.path.join(temp_dir, 'requirements.txt')
with open(req_file_name, 'w') as req_file:
self.write(req_file)
subprocess.check_call([sys.executable, '-m', 'pip'] +
pip_general_options +
['install'] + pip_install_options +
['-r', req_file_name])
def main(default_requirement_file: str) -> None:
"""Command line entry point."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--no-act', '-n',
action='store_true',
help="Don't act, just print what will be done")
parser.add_argument('--pip-install-option',
action='append', dest='pip_install_options',
help="Pass this option to pip install")
parser.add_argument('--pip-option',
action='append', dest='pip_general_options',
help="Pass this general option to pip")
parser.add_argument('--user',
action='append_const', dest='pip_install_options',
const='--user',
help="Install to the Python user install directory"
" (short for --pip-install-option --user)")
parser.add_argument('files', nargs='*', metavar='FILE',
help="Requirement files"
" (default: {})" \
.format(default_requirement_file))
options = parser.parse_args()
if not options.files:
options.files = [default_requirement_file]
reqs = Requirements()
for filename in options.files:
reqs.add_file(filename)
reqs.write(sys.stdout)
if not options.no_act:
reqs.install(pip_general_options=options.pip_general_options,
pip_install_options=options.pip_install_options)

View File

@@ -0,0 +1,463 @@
"""Outcome file analysis code.
This module is the bulk of the code of tests/scripts/analyze_outcomes.py
in each consuming branch. The consuming script is expected to derive
the classes with branch-specific customizations such as ignore lists.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import gzip
import lzma
import sys
import traceback
import re
import subprocess
import os
import typing
from . import collect_test_cases
# `ComponentOutcomes` is a named tuple which is defined as:
# ComponentOutcomes(
# successes = {
# "<suite_case>",
# ...
# },
# failures = {
# "<suite_case>",
# ...
# }
# )
# suite_case = "<suite>;<case>"
ComponentOutcomes = typing.NamedTuple('ComponentOutcomes',
[('successes', typing.Set[str]),
('failures', typing.Set[str])])
# `Outcomes` is a representation of the outcomes file,
# which defined as:
# Outcomes = {
# "<component>": ComponentOutcomes,
# ...
# }
Outcomes = typing.Dict[str, ComponentOutcomes]
class Results:
"""Process analysis results."""
def __init__(self,
stderr: bool = True,
log_file: str = '') -> None:
"""Log and count errors.
Log to stderr if stderr is true.
Log to log_file if specified and non-empty.
"""
self.error_count = 0
self.warning_count = 0
self.stderr = stderr
self.log_file = None
if log_file:
self.log_file = open(log_file, 'w', encoding='utf-8')
def new_section(self, fmt, *args, **kwargs):
self._print_line('\n*** ' + fmt + ' ***\n', *args, **kwargs)
def info(self, fmt, *args, **kwargs):
self._print_line('Info: ' + fmt, *args, **kwargs)
def error(self, fmt, *args, **kwargs):
self.error_count += 1
self._print_line('Error: ' + fmt, *args, **kwargs)
def warning(self, fmt, *args, **kwargs):
self.warning_count += 1
self._print_line('Warning: ' + fmt, *args, **kwargs)
def _print_line(self, fmt, *args, **kwargs):
line = (fmt + '\n').format(*args, **kwargs)
if self.stderr:
sys.stderr.write(line)
if self.log_file:
self.log_file.write(line)
def execute_reference_driver_tests(results: Results, ref_component: str, driver_component: str, \
outcome_file: str) -> None:
"""Run the tests specified in ref_component and driver_component. Results
are stored in the output_file and they will be used for the following
coverage analysis"""
results.new_section("Test {} and {}", ref_component, driver_component)
shell_command = "tests/scripts/all.sh --outcome-file " + outcome_file + \
" " + ref_component + " " + driver_component
results.info("Running: {}", shell_command)
ret_val = subprocess.run(shell_command.split(), check=False).returncode
if ret_val != 0:
results.error("failed to run reference/driver components")
TestCaseMatcher = typing.Union[str, typing.Pattern]
# Map test suite names (with the test_suite prefix) to a list of ignored
# test cases. Each element in the list can be either a string or a
# compiled regex (Pattern). Strings only match themselves. Regexes must
# match the full test case description (not just a prefix or other
# substring).
TestCaseSetDescription = typing.Mapping[str, typing.Sequence[TestCaseMatcher]]
class TestCaseSet:
"""A set of test cases, indexed by their test suite."""
#pylint: disable=too-few-public-methods
def __init__(self, description: TestCaseSetDescription) -> None:
"""Construct a set of test cases from a list of matches for each test suite.
"""
# Construct new mutable objects, to avoid mutating the parameter,
# which could be confusing.
self.matchers = {key: list(entries)
for key, entries in description.items()}
def extend(self, description: TestCaseSetDescription) -> None:
"""Add more matchers to this test case set."""
for key, entries in description.items():
self.matchers.setdefault(key, [])
self.matchers[key] += entries
@staticmethod
def _name_matches_pattern(name: str, str_or_re: TestCaseMatcher) -> bool:
"""Check if name matches a pattern, that may be a string or regex.
- If the pattern is a string, name must be equal to match.
- If the pattern is a regex, name must fully match.
"""
# The CI's python is too old for re.Pattern
#if isinstance(str_or_re, re.Pattern):
if not isinstance(str_or_re, str):
return str_or_re.fullmatch(name) is not None
else:
return str_or_re == name
def _suite_matchers(self, test_suite: str) -> typing.Iterator[TestCaseMatcher]:
"""Generate the matcher list for the specified test suite."""
if test_suite in self.matchers:
yield from self.matchers[test_suite]
pos = test_suite.find('.')
if pos != -1:
base_test_suite = test_suite[:pos]
if base_test_suite in self.matchers:
yield from self.matchers[base_test_suite]
def contains(self, test_suite: str, test_string: str) -> bool:
"""Check if the specified test case is in the set."""
for str_or_re in self._suite_matchers(test_suite):
if self._name_matches_pattern(test_string, str_or_re):
return True
return False
def open_outcome_file(outcome_file: str) -> typing.TextIO:
if outcome_file.endswith('.gz'):
return gzip.open(outcome_file, 'rt', encoding='utf-8')
elif outcome_file.endswith('.xz'):
return lzma.open(outcome_file, 'rt', encoding='utf-8')
else:
return open(outcome_file, 'rt', encoding='utf-8')
def read_outcome_file(outcome_file: str) -> Outcomes:
"""Parse an outcome file and return an outcome collection.
"""
outcomes = {}
with open_outcome_file(outcome_file) as input_file:
for line in input_file:
(_platform, component, suite, case, result, _cause) = line.split(';')
# Note that `component` is not unique. If a test case passes on Linux
# and fails on FreeBSD, it'll end up in both the successes set and
# the failures set.
suite_case = ';'.join([suite, case])
if component not in outcomes:
outcomes[component] = ComponentOutcomes(set(), set())
if result == 'PASS':
outcomes[component].successes.add(suite_case)
elif result == 'FAIL':
outcomes[component].failures.add(suite_case)
return outcomes
class Task:
"""Base class for outcome analysis tasks."""
@staticmethod
def _has_word_re(words: typing.Iterable[str],
exclude: typing.Optional[str] = None) -> typing.Pattern:
"""Construct a regex that matches if any of the words appears.
The occurrence must start and end at a word boundary.
If exclude is specified, strings containing a match for that
regular expression will not match the returned pattern.
"""
exclude_clause = r''
if exclude:
exclude_clause = r'(?!.*' + exclude + ')'
return re.compile(exclude_clause +
r'.*\b(?:' + r'|'.join(words) + r')\b.*',
re.DOTALL)
def __init__(self, options) -> None:
"""Pass command line options to the tasks.
Each task decides which command line options it cares about.
"""
pass
def section_name(self) -> str:
"""The section name to use in results."""
raise NotImplementedError
def run(self, results: Results, outcomes: Outcomes):
"""Run the analysis on the specified outcomes.
Signal errors via the results objects
"""
raise NotImplementedError
class CoverageTask(Task):
"""Analyze test coverage."""
# Test cases whose suite and description are matched by an entry in
# UNCOVERED_TESTS are expected to be never executed.
# Tests matched by IGNORED_TESTS are ignored entirely.
# All other test cases are expected to be executed at least once.
UNCOVERED_TESTS: TestCaseSetDescription = {}
IGNORED_TESTS: TestCaseSetDescription = {}
def __init__(self, options) -> None:
super().__init__(options)
self.full_coverage = options.full_coverage #type: bool
self.uncovered_tests = TestCaseSet(self.UNCOVERED_TESTS)
self.ignored_tests = TestCaseSet(self.IGNORED_TESTS)
@staticmethod
def section_name() -> str:
return "Analyze coverage"
def note_ignored_test(self, results: Results,
test_suite: str, test_description: str) -> None:
# pylint: disable=no-self-use # derived classes may need self
"""This method runs for each test case that's available and ignored."""
results.info('Test case was ignored: {};{}',
test_suite, test_description)
def run(self, results: Results, outcomes: Outcomes) -> None:
"""Check that all available test cases are executed at least once."""
# Make sure that the generated data files are present (and up-to-date).
# This allows analyze_outcomes.py to run correctly on a fresh Git
# checkout.
cp = subprocess.run(['make', 'generated_files'],
cwd='tests',
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
check=False)
if cp.returncode != 0:
sys.stderr.write(cp.stdout.decode('utf-8'))
results.error("Failed \"make generated_files\" in tests. "
"Coverage analysis may be incorrect.")
available = collect_test_cases.collect_available_test_cases()
for suite_case in available:
hit = any(suite_case in comp_outcomes.successes or
suite_case in comp_outcomes.failures
for comp_outcomes in outcomes.values())
(test_suite, test_description) = suite_case.split(';')
ignored = self.ignored_tests.contains(test_suite, test_description)
if ignored:
self.note_ignored_test(results, test_suite, test_description)
continue
uncovered = self.uncovered_tests.contains(test_suite, test_description)
if not hit and not uncovered:
if self.full_coverage:
results.error('Test case not executed: {}', suite_case)
else:
results.warning('Test case not executed: {}', suite_case)
elif hit and uncovered:
# If a test case is no longer always skipped, we should remove
# it from the ignore list.
if self.full_coverage:
results.error(
'Test case was executed but marked as uncovered for coverage: {}',
suite_case)
else:
results.warning(
'Test case was executed but marked as uncovered for coverage: {}',
suite_case)
class DriverVSReference(Task):
"""Compare outcomes from testing with and without a driver.
There are 2 options to use analyze_driver_vs_reference_xxx locally:
1. Run tests and then analysis:
- tests/scripts/all.sh --outcome-file "$PWD/out.csv" <component_ref> <component_driver>
- tests/scripts/analyze_outcomes.py out.csv analyze_driver_vs_reference_xxx
2. Let this script run both automatically:
- tests/scripts/analyze_outcomes.py out.csv analyze_driver_vs_reference_xxx
"""
# Override the following in child classes.
# Configuration name (all.sh component) used as the reference.
REFERENCE = ''
# Configuration name (all.sh component) used as the driver.
DRIVER = ''
# Ignored test suites (without the test_suite_ prefix).
IGNORED_SUITES = [] #type: typing.Sequence[str]
# Ignored test cases. Despite the name, these test cases are not
# completely ignored: they must be skipped by driver tests. If they
# are not skipped, this indicates a spurious entry and the analysis will
# complain.
IGNORED_TESTS: TestCaseSetDescription = {}
def __init__(self, options) -> None:
super().__init__(options)
self.ignored_suites = frozenset('test_suite_' + x
for x in self.IGNORED_SUITES)
self.ignored_tests = TestCaseSet(self.IGNORED_TESTS)
def section_name(self) -> str:
return f"Analyze driver {self.DRIVER} vs reference {self.REFERENCE}"
def run(self, results: Results, outcomes: Outcomes) -> None:
"""Check that all tests passing in the driver component are also
passing in the corresponding reference component.
Skip:
- full test suites provided in ignored_suites list
- only some specific test inside a test suite, for which the corresponding
output string is provided
"""
ref_outcomes = outcomes.get("component_" + self.REFERENCE)
driver_outcomes = outcomes.get("component_" + self.DRIVER)
if ref_outcomes is None or driver_outcomes is None:
results.error("required components are missing: bad outcome file?")
return
if not ref_outcomes.successes:
results.error("no passing test in reference component: bad outcome file?")
return
for suite_case in ref_outcomes.successes:
# suite_case is like "test_suite_foo.bar;Description of test case"
(full_test_suite, test_string) = suite_case.split(';')
test_suite = full_test_suite.split('.')[0] # retrieve main part of test suite name
# Immediately skip fully-ignored test suites
if test_suite in self.ignored_suites or \
full_test_suite in self.ignored_suites:
continue
# For ignored test cases inside test suites, just remember and:
# don't issue an error if they're skipped with drivers,
# but issue an error if they're not (means we have a bad entry).
ignored = self.ignored_tests.contains(full_test_suite, test_string)
if not ignored and not suite_case in driver_outcomes.successes:
results.error("SKIP/FAIL -> PASS: {}", suite_case)
if ignored and suite_case in driver_outcomes.successes:
results.error("uselessly ignored: {}", suite_case)
# Set this to False if a consuming branch can't achieve full test coverage
# in its default CI run.
FULL_COVERAGE_BY_DEFAULT = True
def main(known_tasks: typing.Dict[str, typing.Type[Task]]) -> None:
try:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('outcomes', metavar='OUTCOMES.CSV',
help='Outcome file to analyze (can be .gz or .xz)')
parser.add_argument('specified_tasks', default='all', nargs='?',
help='Analysis to be done. By default, run all tasks. '
'With one or more TASK, run only those. '
'TASK can be the name of a single task or '
'comma/space-separated list of tasks. ')
parser.add_argument('--allow-partial-coverage', action='store_false',
dest='full_coverage', default=FULL_COVERAGE_BY_DEFAULT,
help=("Only warn if a test case is skipped in all components" +
(" (default)" if not FULL_COVERAGE_BY_DEFAULT else "") +
". Only used by the 'analyze_coverage' task."))
parser.add_argument('--list', action='store_true',
help='List all available tasks and exit.')
parser.add_argument('--log-file',
default='tests/analyze_outcomes.log',
help='Log file (default: tests/analyze_outcomes.log;'
' empty means no log file)')
parser.add_argument('--require-full-coverage', action='store_true',
dest='full_coverage', default=FULL_COVERAGE_BY_DEFAULT,
help=("Require all available test cases to be executed" +
(" (default)" if FULL_COVERAGE_BY_DEFAULT else "") +
". Only used by the 'analyze_coverage' task."))
options = parser.parse_args()
if options.list:
for task_name in known_tasks:
print(task_name)
sys.exit(0)
main_results = Results(log_file=options.log_file)
if options.specified_tasks == 'all':
tasks_list = list(known_tasks.keys())
else:
tasks_list = re.split(r'[, ]+', options.specified_tasks)
for task_name in tasks_list:
if task_name not in known_tasks:
sys.stderr.write('invalid task: {}\n'.format(task_name))
sys.exit(2)
# If the outcome file exists, parse it once and share the result
# among tasks to improve performance.
# Otherwise, it will be generated by execute_reference_driver_tests.
if not os.path.exists(options.outcomes):
if len(tasks_list) > 1:
sys.stderr.write("mutiple tasks found, please provide a valid outcomes file.\n")
sys.exit(2)
task_name = tasks_list[0]
task_class = known_tasks[task_name]
if not issubclass(task_class, DriverVSReference):
sys.stderr.write("please provide valid outcomes file for {}.\n".format(task_name))
sys.exit(2)
# mypy isn't smart enough to know that REFERENCE and DRIVER
# are *class* attributes of all classes derived from
# DriverVSReference. (It would be smart enough if we had an
# instance of task_class, but we can't construct an instance
# until we have the outcome data, so at this point we only
# have the class.) So we use indirection to access the class
# attributes.
execute_reference_driver_tests(main_results,
getattr(task_class, 'REFERENCE'),
getattr(task_class, 'DRIVER'),
options.outcomes)
outcomes = read_outcome_file(options.outcomes)
for task_name in tasks_list:
task_constructor = known_tasks[task_name]
task_instance = task_constructor(options)
main_results.new_section(task_instance.section_name())
task_instance.run(main_results, outcomes)
main_results.info("Overall results: {} warnings and {} errors",
main_results.warning_count, main_results.error_count)
sys.exit(0 if (main_results.error_count == 0) else 1)
except Exception: # pylint: disable=broad-except
# Print the backtrace and exit explicitly with our chosen status.
traceback.print_exc()
sys.exit(120)

View File

@@ -0,0 +1,189 @@
"""Run the PSA Crypto API compliance test suite.
Clone the repo and check out the commit specified by PSA_ARCH_TEST_REPO and PSA_ARCH_TEST_REF,
then compile and run the test suite. The clone is stored at <repository root>/psa-arch-tests.
Known defects in either the test suite or mbedtls / TF-PSA-Crypto - identified by their test
number - are ignored, while unexpected failures AND successes are reported as errors, to help
keep the list of known defects as up to date as possible.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import glob
import os
import re
import shutil
import subprocess
import sys
from typing import List, Optional
from pathlib import Path
from . import build_tree
PSA_ARCH_TESTS_REPO = 'https://github.com/ARM-software/psa-arch-tests.git'
#pylint: disable=too-many-branches,too-many-statements,too-many-locals
def test_compliance(library_build_dir: str,
psa_arch_tests_ref: str,
patch_files: List[str],
expected_failures: List[int]) -> int:
"""Check out and run compliance tests.
library_build_dir: path where our library will be built.
psa_arch_tests_ref: tag or sha to use for the arch-tests.
patch: patch to apply to the arch-tests with ``patch -p1``.
expected_failures: default list of expected failures.
"""
root_dir = os.getcwd()
install_dir = Path(library_build_dir + "/install_dir").resolve()
tmp_env = os.environ
tmp_env['CC'] = 'gcc'
subprocess.check_call(['cmake', '.', '-GUnix Makefiles',
'-B' + library_build_dir,
'-DCMAKE_INSTALL_PREFIX=' + str(install_dir)],
env=tmp_env)
subprocess.check_call(['cmake', '--build', library_build_dir, '--target', 'install'])
if build_tree.is_mbedtls_3_6():
crypto_library_path = install_dir.joinpath("lib/libmbedcrypto.a")
else:
crypto_library_path = install_dir.joinpath("lib/libtfpsacrypto.a")
psa_arch_tests_dir = 'psa-arch-tests'
os.makedirs(psa_arch_tests_dir, exist_ok=True)
try:
os.chdir(psa_arch_tests_dir)
# Reuse existing local clone
subprocess.check_call(['git', 'init'])
subprocess.check_call(['git', 'fetch', PSA_ARCH_TESTS_REPO, psa_arch_tests_ref])
subprocess.check_call(['git', 'checkout', '--force', 'FETCH_HEAD'])
if patch_files:
subprocess.check_call(['git', 'reset', '--hard'])
for patch_file in patch_files:
with open(os.path.join(root_dir, patch_file), 'rb') as patch:
subprocess.check_call(['patch', '-p1'],
stdin=patch)
build_dir = 'api-tests/build'
try:
shutil.rmtree(build_dir)
except FileNotFoundError:
pass
os.mkdir(build_dir)
os.chdir(build_dir)
#pylint: disable=bad-continuation
subprocess.check_call([
'cmake', '..',
'-GUnix Makefiles',
'-DTARGET=tgt_dev_apis_stdc',
'-DTOOLCHAIN=HOST_GCC',
'-DSUITE=CRYPTO',
'-DPSA_CRYPTO_LIB_FILENAME={}'.format(str(crypto_library_path)),
'-DPSA_INCLUDE_PATHS=' + str(install_dir.joinpath("include"))
])
subprocess.check_call(['cmake', '--build', '.'])
proc = subprocess.Popen(['./psa-arch-tests-crypto'],
bufsize=1, stdout=subprocess.PIPE, universal_newlines=True)
test_re = re.compile(
'^TEST: (?P<test_num>[0-9]*)|'
'^TEST RESULT: (?P<test_result>FAILED|PASSED)'
)
test = -1
unexpected_successes = expected_failures.copy()
expected_failures.clear()
unexpected_failures = [] # type: List[int]
if proc.stdout is None:
return 1
for line in proc.stdout:
print(line, end='')
match = test_re.match(line)
if match is not None:
groupdict = match.groupdict()
test_num = groupdict['test_num']
if test_num is not None:
test = int(test_num)
elif groupdict['test_result'] == 'FAILED':
try:
unexpected_successes.remove(test)
expected_failures.append(test)
print('Expected failure, ignoring')
except KeyError:
unexpected_failures.append(test)
print('ERROR: Unexpected failure')
elif test in unexpected_successes:
print('ERROR: Unexpected success')
proc.wait()
print()
print('***** test_psa_compliance.py report ******')
print()
print('Expected failures:', ', '.join(str(i) for i in expected_failures))
print('Unexpected failures:', ', '.join(str(i) for i in unexpected_failures))
print('Unexpected successes:', ', '.join(str(i) for i in sorted(unexpected_successes)))
print()
if unexpected_successes or unexpected_failures:
if unexpected_successes:
print('Unexpected successes encountered.')
print('Please remove the corresponding tests from '
'EXPECTED_FAILURES in tests/scripts/compliance_test.py')
print()
print('FAILED')
return 1
else:
print('SUCCESS')
return 0
finally:
os.chdir(root_dir)
def main(psa_arch_tests_ref: str,
expected_failures: Optional[List[int]] = None) -> None:
"""Command line entry point.
psa_arch_tests_ref: tag or sha to use for the arch-tests.
expected_failures: default list of expected failures.
"""
build_dir = 'out_of_source_build'
default_patch_directory = os.path.join(build_tree.guess_project_root(),
'scripts/data_files/psa-arch-tests')
# pylint: disable=invalid-name
parser = argparse.ArgumentParser()
parser.add_argument('--build-dir', nargs=1,
help='path to Mbed TLS / TF-PSA-Crypto build directory')
parser.add_argument('--expected-failures', nargs='+',
help='''set the list of test codes which are expected to fail
from the command line. If omitted the list given by
EXPECTED_FAILURES (inside the script) is used.''')
parser.add_argument('--patch-directory', nargs=1,
default=default_patch_directory,
help='Directory containing patches (*.patch) to apply to psa-arch-tests')
args = parser.parse_args()
if args.build_dir is not None:
build_dir = args.build_dir[0]
if expected_failures is None:
expected_failures = []
if args.expected_failures is not None:
expected_failures_list = [int(i) for i in args.expected_failures]
else:
expected_failures_list = expected_failures
if args.patch_directory:
patch_file_glob = os.path.join(args.patch_directory, '*.patch')
patch_files = sorted(glob.glob(patch_file_glob))
else:
patch_files = []
sys.exit(test_compliance(build_dir,
psa_arch_tests_ref,
patch_files,
expected_failures_list))

View File

@@ -0,0 +1,162 @@
"""Collect information about PSA cryptographic mechanisms.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import re
from collections import OrderedDict
from typing import List, Optional
from . import build_tree
from . import macro_collector
class Information:
"""Gather information about PSA constructors."""
def __init__(self) -> None:
self.constructors = self.read_psa_interface()
@staticmethod
def remove_unwanted_macros(
constructors: macro_collector.PSAMacroEnumerator
) -> None:
"""Remove constructors that should be exckuded from systematic testing."""
# Mbed TLS does not support finite-field DSA, but 3.6 defines DSA
# identifiers for historical reasons.
# Mbed TLS and TF-PSA-Crypto 1.0 do not support SPAKE2+, although
# TF-PSA-Crypto 1.0 defines SPAKE2+ identifiers to be able to build
# the psa-arch-tests compliance test suite.
#
# Don't attempt to generate any related test case.
# The corresponding test cases would be commented out anyway,
# but for these types, we don't have enough support in the test scripts
# to generate these test cases.
constructors.key_types.discard('PSA_KEY_TYPE_DSA_KEY_PAIR')
constructors.key_types.discard('PSA_KEY_TYPE_DSA_PUBLIC_KEY')
constructors.key_types.discard('PSA_KEY_TYPE_SPAKE2P_KEY_PAIR')
constructors.key_types.discard('PSA_KEY_TYPE_SPAKE2P_PUBLIC_KEY')
def read_psa_interface(self) -> macro_collector.PSAMacroEnumerator:
"""Return the list of known key types, algorithms, etc."""
constructors = macro_collector.InputsForTest()
if build_tree.looks_like_root('.'):
if build_tree.looks_like_mbedtls_root('.') and \
(not build_tree.is_mbedtls_3_6()):
header_file_names = ['tf-psa-crypto/include/psa/crypto_values.h',
'tf-psa-crypto/include/psa/crypto_extra.h']
test_suites = ['tf-psa-crypto/tests/suites/test_suite_psa_crypto_metadata.data']
else:
header_file_names = ['include/psa/crypto_values.h',
'include/psa/crypto_extra.h']
test_suites = ['tests/suites/test_suite_psa_crypto_metadata.data']
for header_file_name in header_file_names:
constructors.parse_header(header_file_name)
for test_cases in test_suites:
constructors.parse_test_cases(test_cases)
self.remove_unwanted_macros(constructors)
constructors.gather_arguments()
return constructors
def psa_want_symbol(name: str, prefix: Optional[str] = None) -> str:
"""Return the PSA_WANT_xxx symbol associated with a PSA crypto feature.
You can use an altenative `prefix`, e.g. 'MBEDTLS_PSA_BUILTIN_'
when specifically testing builtin implementations.
"""
if prefix is None:
prefix = 'PSA_WANT_'
if name.startswith('PSA_'):
return prefix + name[4:]
else:
raise ValueError('Unable to determine the PSA_WANT_ symbol for ' + name)
def finish_family_dependency(dep: str, bits: int) -> str:
"""Finish dep if it's a family dependency symbol prefix.
A family dependency symbol prefix is a PSA_WANT_ symbol that needs to be
qualified by the key size. If dep is such a symbol, finish it by adjusting
the prefix and appending the key size. Other symbols are left unchanged.
"""
return re.sub(r'_FAMILY_(.*)', r'_\1_' + str(bits), dep)
def finish_family_dependencies(dependencies: List[str], bits: int) -> List[str]:
"""Finish any family dependency symbol prefixes.
Apply `finish_family_dependency` to each element of `dependencies`.
"""
return [finish_family_dependency(dep, bits) for dep in dependencies]
SYMBOLS_WITHOUT_DEPENDENCY = frozenset([
'PSA_ALG_AEAD_WITH_AT_LEAST_THIS_LENGTH_TAG', # modifier, only in policies
'PSA_ALG_AEAD_WITH_SHORTENED_TAG', # modifier
'PSA_ALG_ANY_HASH', # only in policies
'PSA_ALG_AT_LEAST_THIS_LENGTH_MAC', # modifier, only in policies
'PSA_ALG_KEY_AGREEMENT', # chaining
'PSA_ALG_TRUNCATED_MAC', # modifier
])
def automatic_dependencies(*expressions: str,
prefix: Optional[str] = None) -> List[str]:
"""Infer dependencies of a test case by looking for PSA_xxx symbols.
The arguments are strings which should be C expressions. Do not use
string literals or comments as this function is not smart enough to
skip them.
`prefix`: prefix to use in dependencies. Defaults to ``'PSA_WANT_'``.
Use ``'MBEDTLS_PSA_BUILTIN_'`` when specifically testing
builtin implementations.
"""
used = set()
for expr in expressions:
used.update(re.findall(r'PSA_(?:ALG|ECC_FAMILY|DH_FAMILY|KEY_TYPE)_\w+', expr))
used.difference_update(SYMBOLS_WITHOUT_DEPENDENCY)
return sorted(psa_want_symbol(name, prefix=prefix) for name in used)
# Define set of regular expressions and dependencies to optionally append
# extra dependencies for test case based on key description.
# Skip AES test cases which require 192- or 256-bit key
# if MBEDTLS_AES_ONLY_128_BIT_KEY_LENGTH defined
AES_128BIT_ONLY_DEP_REGEX = re.compile(r'AES\s(192|256)')
AES_128BIT_ONLY_DEP = ['!MBEDTLS_AES_ONLY_128_BIT_KEY_LENGTH']
# Skip AES/ARIA/CAMELLIA test cases which require decrypt operation in ECB mode
# if MBEDTLS_BLOCK_CIPHER_NO_DECRYPT enabled.
ECB_NO_PADDING_DEP_REGEX = re.compile(r'(AES|ARIA|CAMELLIA).*ECB_NO_PADDING')
ECB_NO_PADDING_DEP = ['!MBEDTLS_BLOCK_CIPHER_NO_DECRYPT']
DEPENDENCY_FROM_DESCRIPTION = OrderedDict()
DEPENDENCY_FROM_DESCRIPTION[AES_128BIT_ONLY_DEP_REGEX] = AES_128BIT_ONLY_DEP
DEPENDENCY_FROM_DESCRIPTION[ECB_NO_PADDING_DEP_REGEX] = ECB_NO_PADDING_DEP
def generate_deps_from_description(
description: str
) -> List[str]:
"""Return additional dependencies based on test case description and REGEX.
"""
dep_list = []
for regex, deps in DEPENDENCY_FROM_DESCRIPTION.items():
if re.search(regex, description):
dep_list += deps
return dep_list
def tweak_key_pair_dependency(dep: str, usages: List[str]) -> List[str]:
"""
This helper function add the proper suffix to PSA_WANT_KEY_TYPE_xxx_KEY_PAIR
symbols according to the required usage.
"""
if dep.endswith('KEY_PAIR'):
return [dep + '_' + usage for usage in usages]
return [dep]
def fix_key_pair_dependencies(dep_list: List[str], usages: List[str]) -> List[str]:
new_list = [new_deps
for dep in dep_list
for new_deps in tweak_key_pair_dependency(dep, usages)]
return new_list

View File

@@ -0,0 +1,221 @@
"""Knowledge about the PSA key store as implemented in Mbed TLS.
Note that if you need to make a change that affects how keys are
stored, this may indicate that the key store is changing in a
backward-incompatible way! Think carefully about backward compatibility
before changing how test data is constructed or validated.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import re
import struct
from typing import Dict, List, Optional, Set, Union
import unittest
from . import c_build_helper
from . import build_tree
class Expr:
"""Representation of a C expression with a known or knowable numerical value."""
def __init__(self, content: Union[int, str]):
if isinstance(content, int):
digits = 8 if content > 0xffff else 4
self.string = '{0:#0{1}x}'.format(content, digits + 2)
self.value_if_known = content #type: Optional[int]
else:
self.string = content
self.unknown_values.add(self.normalize(content))
self.value_if_known = None
value_cache = {} #type: Dict[str, int]
"""Cache of known values of expressions."""
unknown_values = set() #type: Set[str]
"""Expressions whose values are not present in `value_cache` yet."""
def update_cache(self) -> None:
"""Update `value_cache` for expressions registered in `unknown_values`."""
expressions = sorted(self.unknown_values)
# Temporary, while Mbed TLS does not just rely on the TF-PSA-Crypto
# build system to build its crypto library. When it does, the first
# case can just be removed.
if build_tree.looks_like_root('.'):
includes = ['include']
if build_tree.looks_like_tf_psa_crypto_root('.'):
includes.append('drivers/builtin/include')
includes.append('drivers/everest/include')
includes.append('drivers/everest/include/tf-psa-crypto/private/')
includes.append('drivers/pqcp/include')
elif not build_tree.is_mbedtls_3_6():
includes.append('tf-psa-crypto/include')
includes.append('tf-psa-crypto/drivers/builtin/include')
includes.append('tf-psa-crypto/drivers/everest/include')
includes.append('tf-psa-crypto/drivers/everest/include/tf-psa-crypto/private/')
includes.append('tf-psa-crypto/drivers/pqcp/include')
values = c_build_helper.get_c_expression_values(
'unsigned long', '%lu',
expressions,
header="""
#include <psa/crypto.h>
""",
include_path=includes) #type: List[str]
for e, v in zip(expressions, values):
self.value_cache[e] = int(v, 0)
self.unknown_values.clear()
@staticmethod
def normalize(string: str) -> str:
"""Put the given C expression in a canonical form.
This function is only intended to give correct results for the
relatively simple kind of C expression typically used with this
module.
"""
return re.sub(r'\s+', r'', string)
def value(self) -> int:
"""Return the numerical value of the expression."""
if self.value_if_known is None:
if re.match(r'([0-9]+|0x[0-9a-f]+)\Z', self.string, re.I):
return int(self.string, 0)
normalized = self.normalize(self.string)
if normalized not in self.value_cache:
self.update_cache()
self.value_if_known = self.value_cache[normalized]
return self.value_if_known
Exprable = Union[str, int, Expr]
"""Something that can be converted to a C expression with a known numerical value."""
def as_expr(thing: Exprable) -> Expr:
"""Return an `Expr` object for `thing`.
If `thing` is already an `Expr` object, return it. Otherwise build a new
`Expr` object from `thing`. `thing` can be an integer or a string that
contains a C expression.
"""
if isinstance(thing, Expr):
return thing
else:
return Expr(thing)
class Key:
"""Representation of a PSA crypto key object and its storage encoding.
"""
LATEST_VERSION = 0
"""The latest version of the storage format."""
def __init__(self, *,
version: Optional[int] = None,
id: Optional[int] = None, #pylint: disable=redefined-builtin
lifetime: Exprable = 'PSA_KEY_LIFETIME_PERSISTENT',
type: Exprable, #pylint: disable=redefined-builtin
bits: int,
usage: Exprable, alg: Exprable, alg2: Exprable,
material: bytes #pylint: disable=used-before-assignment
) -> None:
self.version = self.LATEST_VERSION if version is None else version
self.id = id #pylint: disable=invalid-name #type: Optional[int]
self.lifetime = as_expr(lifetime) #type: Expr
self.type = as_expr(type) #type: Expr
self.bits = bits #type: int
self.usage = as_expr(usage) #type: Expr
self.alg = as_expr(alg) #type: Expr
self.alg2 = as_expr(alg2) #type: Expr
self.material = material #type: bytes
MAGIC = b'PSA\000KEY\000'
@staticmethod
def pack(
fmt: str,
*args: Union[int, Expr]
) -> bytes: #pylint: disable=used-before-assignment
"""Pack the given arguments into a byte string according to the given format.
This function is similar to `struct.pack`, but with the following differences:
* All integer values are encoded with standard sizes and in
little-endian representation. `fmt` must not include an endianness
prefix.
* Arguments can be `Expr` objects instead of integers.
* Only integer-valued elements are supported.
"""
return struct.pack('<' + fmt, # little-endian, standard sizes
*[arg.value() if isinstance(arg, Expr) else arg
for arg in args])
def bytes(self) -> bytes:
"""Return the representation of the key in storage as a byte array.
This is the content of the PSA storage file. When PSA storage is
implemented over stdio files, this does not include any wrapping made
by the PSA-storage-over-stdio-file implementation.
Note that if you need to make a change in this function,
this may indicate that the key store is changing in a
backward-incompatible way! Think carefully about backward
compatibility before making any change here.
"""
header = self.MAGIC + self.pack('L', self.version)
if self.version == 0:
attributes = self.pack('LHHLLL',
self.lifetime, self.type, self.bits,
self.usage, self.alg, self.alg2)
material = self.pack('L', len(self.material)) + self.material
else:
raise NotImplementedError
return header + attributes + material
def hex(self) -> str:
"""Return the representation of the key as a hexadecimal string.
This is the hexadecimal representation of `self.bytes`.
"""
return self.bytes().hex()
def location_value(self) -> int:
"""The numerical value of the location encoded in the key's lifetime."""
return self.lifetime.value() >> 8
class TestKey(unittest.TestCase):
# pylint: disable=line-too-long
"""A few smoke tests for the functionality of the `Key` class."""
def test_numerical(self):
key = Key(version=0,
id=1, lifetime=0x00000001,
type=0x2400, bits=128,
usage=0x00000300, alg=0x05500200, alg2=0x04c01000,
material=b'@ABCDEFGHIJKLMNO')
expected_hex = '505341004b45590000000000010000000024800000030000000250050010c00410000000404142434445464748494a4b4c4d4e4f'
self.assertEqual(key.bytes(), bytes.fromhex(expected_hex))
self.assertEqual(key.hex(), expected_hex)
def test_names(self):
length = 0xfff8 // 8 # PSA_MAX_KEY_BITS in bytes
key = Key(version=0,
id=1, lifetime='PSA_KEY_LIFETIME_PERSISTENT',
type='PSA_KEY_TYPE_RAW_DATA', bits=length*8,
usage=0, alg=0, alg2=0,
material=b'\x00' * length)
expected_hex = '505341004b45590000000000010000000110f8ff000000000000000000000000ff1f0000' + '00' * length
self.assertEqual(key.bytes(), bytes.fromhex(expected_hex))
self.assertEqual(key.hex(), expected_hex)
def test_defaults(self):
key = Key(type=0x1001, bits=8,
usage=0, alg=0, alg2=0,
material=b'\x2a')
expected_hex = '505341004b455900000000000100000001100800000000000000000000000000010000002a'
self.assertEqual(key.bytes(), bytes.fromhex(expected_hex))
self.assertEqual(key.hex(), expected_hex)

View File

@@ -0,0 +1,205 @@
"""Generate test cases for PSA API calls, with automatic dependencies.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import os
import re
from typing import FrozenSet, List, Optional, Set
from . import build_tree
from . import psa_information
from . import test_case
# Skip test cases for which the dependency symbols are not defined.
# We assume that this means that a required mechanism is not implemented.
# Note that if we erroneously skip generating test cases for
# mechanisms that are not implemented, this should be caught
# by the NOT_SUPPORTED test cases generated by generate_psa_tests.py
# in test_suite_psa_crypto_not_supported and test_suite_psa_crypto_op_fail:
# those emit tests with negative dependencies, which will not be skipped here.
def read_implemented_dependencies(acc: Set[str], filename: str) -> None:
with open(filename) as input_stream:
for line in input_stream:
for symbol in re.findall(r'\bPSA_WANT_\w+\b', line):
acc.add(symbol)
_implemented_dependencies = None #type: Optional[FrozenSet[str]] #pylint: disable=invalid-name
def find_dependencies_not_implemented(dependencies: List[str]) -> List[str]:
"""List the dependencies that are not implemented."""
global _implemented_dependencies #pylint: disable=global-statement,invalid-name
if _implemented_dependencies is None:
# Temporary, while Mbed TLS does not just rely on the TF-PSA-Crypto
# build system to build its crypto library. When it does, the first
# case can just be removed.
if build_tree.looks_like_root('.'):
if build_tree.looks_like_mbedtls_root('.') and \
(not build_tree.is_mbedtls_3_6()):
include_dir = 'tf-psa-crypto/include'
else:
include_dir = 'include'
acc = set() #type: Set[str]
for filename in [
'psa/crypto_config.h',
'psa/crypto_adjust_config_synonyms.h',
'tf-psa-crypto/private/crypto_adjust_config_synonyms.h',
]:
path = os.path.join(build_tree.guess_project_root(),
include_dir,
filename)
if os.path.exists(path):
read_implemented_dependencies(acc, path)
_implemented_dependencies = frozenset(acc)
return [dep
for dep in dependencies
if (dep not in _implemented_dependencies and
dep.startswith('PSA_WANT'))]
class TestCase(test_case.TestCase):
"""A PSA test case with automatically inferred dependencies.
For mechanisms like ECC curves where the support status includes
the key bit-size, this class assumes that only one bit-size is
involved in a given test case.
"""
def __init__(self, dependency_prefix: Optional[str] = None) -> None:
"""Construct a test case for a PSA Crypto API call.
`dependency_prefix`: prefix to use in dependencies. Defaults to
``'PSA_WANT_'``. Use ``'MBEDTLS_PSA_BUILTIN_'``
when specifically testing builtin implementations.
"""
super().__init__()
del self.dependencies
self.manual_dependencies = [] #type: List[str]
self.automatic_dependencies = set() #type: Set[str]
self.dependency_prefix = dependency_prefix #type: Optional[str]
self.negated_dependencies = set() #type: Set[str]
self.key_bits = None #type: Optional[int]
self.key_pair_usage = None #type: Optional[List[str]]
def set_key_bits(self, key_bits: Optional[int]) -> None:
"""Use the given key size for automatic dependency generation.
Call this function before set_arguments() if relevant.
This is only relevant for ECC and DH keys. For other key types,
this information is ignored.
"""
self.key_bits = key_bits
def set_key_pair_usage(self, key_pair_usage: Optional[List[str]]) -> None:
"""Use the given suffixes for key pair dependencies.
Call this function before set_arguments() if relevant.
This is only relevant for key pair types. For other key types,
this information is ignored.
"""
self.key_pair_usage = key_pair_usage
def infer_dependencies(self, arguments: List[str]) -> List[str]:
"""Infer dependencies based on the test case arguments."""
dependencies = psa_information.automatic_dependencies(*arguments,
prefix=self.dependency_prefix)
if self.key_bits is not None:
dependencies = psa_information.finish_family_dependencies(dependencies,
self.key_bits)
if self.key_pair_usage is not None:
dependencies = psa_information.fix_key_pair_dependencies(dependencies,
self.key_pair_usage)
if 'PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_GENERATE' in dependencies and \
'PSA_WANT_KEY_TYPE_RSA_KEY_PAIR_GENERATE' not in self.negated_dependencies and \
self.key_bits is not None:
size_dependency = ('PSA_VENDOR_RSA_GENERATE_MIN_KEY_BITS <= ' +
str(self.key_bits))
dependencies.append(size_dependency)
return dependencies
def assumes_not_supported(self, name: str) -> None:
"""Negate the given mechanism for automatic dependency generation.
`name` can be either a dependency symbol (``PSA_WANT_xxx``) or
a mechanism name (``PSA_KEY_TYPE_xxx``, etc.).
Call this function before set_arguments() for a test case that should
run if the given mechanism is not supported.
Call modifiers such as set_key_bits() and set_key_pair_usage() before
calling this method, if applicable.
A mechanism is a PSA_XXX symbol, e.g. PSA_KEY_TYPE_AES, PSA_ALG_HMAC,
etc. For mechanisms like ECC curves where the support status includes
the key bit-size, this class assumes that only one bit-size is
involved in a given test case.
"""
if name.startswith('PSA_WANT_'):
self.negated_dependencies.add(name)
return
if name == 'PSA_KEY_TYPE_RSA_KEY_PAIR' and \
self.key_bits is not None and \
self.key_pair_usage == ['GENERATE']:
# When RSA key pair generation is not supported, it could be
# due to the specific key size is out of range, or because
# RSA key pair generation itself is not supported. Assume the
# latter.
dep = psa_information.psa_want_symbol(name, prefix=self.dependency_prefix)
self.negated_dependencies.add(dep + '_GENERATE')
return
dependencies = self.infer_dependencies([name])
# * If we have more than one dependency to negate, the result would
# say that all of the dependencies are disabled, which is not
# a desirable outcome: the negation of (A and B) is (!A or !B),
# not (!A and !B).
# * If we have no dependency to negate, the result wouldn't be a
# not-supported case.
# Assert that we don't reach either such case.
assert len(dependencies) == 1
self.negated_dependencies.add(dependencies[0])
def set_arguments(self, arguments: List[str]) -> None:
"""Set test case arguments and automatically infer dependencies."""
super().set_arguments(arguments)
dependencies = self.infer_dependencies(arguments)
for i in range(len(dependencies)): #pylint: disable=consider-using-enumerate
if dependencies[i] in self.negated_dependencies:
dependencies[i] = '!' + dependencies[i]
self.skip_if_any_not_implemented(dependencies)
self.automatic_dependencies.update(dependencies)
def set_dependencies(self, dependencies: List[str]) -> None:
"""Override any previously added automatic or manual dependencies.
Also override any previous instruction to skip the test case.
"""
self.manual_dependencies = dependencies
self.automatic_dependencies.clear()
self.skip_reasons = []
def add_dependencies(self, dependencies: List[str]) -> None:
"""Add manual dependencies."""
self.manual_dependencies += dependencies
def get_dependencies(self) -> List[str]:
# Make the output independent of the order in which the dependencies
# are calculated by the script. Also avoid duplicates. This makes
# the output robust with respect to refactoring of the scripts.
dependencies = set(self.manual_dependencies)
dependencies.update(self.automatic_dependencies)
return sorted(dependencies)
def skip_if_any_not_implemented(self, dependencies: List[str]) -> None:
"""Skip the test case if any of the given dependencies is not implemented."""
not_implemented = find_dependencies_not_implemented(dependencies)
for dep in not_implemented:
self.skip_because('not implemented: ' + dep)

View File

@@ -0,0 +1,352 @@
"""Common code for generating the test code to validate mbedtls_ssl_session_reset().
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import enum
import os
import re
import sys
import typing
import argparse
from typing import Dict, Iterator, List, Tuple, FrozenSet
from . import c_parsing_helper
from . import typing_util
from . import build_tree
class ResetBehavior(enum.Enum):
KEEP = 0 # Kept unchanged before/after the reset
RESET = 1 # Returned to the initial state (which is not necessarily 0)
REALLOCATE = 2 # Pointer that gets reallocated
IGNORE = 3 # Ignored field
SPECIAL = 4
class ElementType(enum.Enum):
SCALAR = 0
POINTER = 1
ARRAY = 2
NAMED_STRUCTURE = 3
IGNORE = 4
SPECIAL = 5
class FieldsInfo(typing.NamedTuple):
"""Expected reset behavior for the fields of the structure."""
rules: Dict[str, ResetBehavior]
special: Dict[str, List[str]]
# The script isn't capable to identify named structures (ex: dtls_srtp_info)
# so we keep an explicit list of them.
named_structures: FrozenSet[str]
class CField():
# pylint: disable=too-few-public-methods
"""Information about one field of a C struct."""
name: str
conditional: List[str]
element_type: ElementType
def __init__(self, name: str, conditionals: List[str], element_type: ElementType):
self.name = name
self.conditionals = conditionals.copy()
self.element_type = element_type
def check_value(self) -> List[str]:
raise Exception(f'Class {self.__class__.__name__} cannot handle entries'
f'of type {self.element_type}')
class CFieldIgnore(CField):
# pylint: disable=too-few-public-methods
"""Explicitly ignored field."""
def check_value(self) -> List[str]:
return [f'/* {self.name} is ignored */']
class CFieldKeep(CField):
# pylint: disable=too-few-public-methods
"""Field kept unchanged."""
def check_value(self) -> List[str]:
if self.element_type == ElementType.SCALAR:
return [f'TEST_EQUAL(before->{self.name}, after->{self.name});']
elif self.element_type == ElementType.POINTER:
return [f'TEST_ASSERT(before->{self.name} == after->{self.name});']
elif self.element_type == ElementType.ARRAY:
return [f'TEST_MEMORY_COMPARE(before->{self.name}, '
f'sizeof(before->{self.name}), after->{self.name}, '
f'sizeof(after->{self.name}));']
elif self.element_type == ElementType.NAMED_STRUCTURE:
return [f'TEST_MEMORY_COMPARE(&(before->{self.name}), '
f'sizeof(before->{self.name}), &(after->{self.name}), '
f'sizeof(after->{self.name}));']
return super().check_value()
class CFieldReset(CField):
# pylint: disable=too-few-public-methods
"""Field returned to the intial state."""
def check_value(self) -> List[str]:
if (self.element_type == ElementType.SCALAR) or (self.element_type == ElementType.POINTER):
return [f'TEST_ASSERT(after->{self.name} == initial.{self.name});']
elif self.element_type == ElementType.ARRAY:
return [f'TEST_MEMORY_COMPARE(after->{self.name}, '
f'sizeof(after->{self.name}), initial.{self.name}, '
f'sizeof(initial.{self.name}));']
elif self.element_type == ElementType.NAMED_STRUCTURE:
return [f'TEST_MEMORY_COMPARE(&(after->{self.name}), '
f'sizeof(after->{self.name}), &(initial.{self.name}), '
f'sizeof(initial.{self.name}));']
return super().check_value()
class CFieldReallocate(CField):
# pylint: disable=too-few-public-methods
"""Pointer (might be) reallocated during reset."""
def check_value(self) -> List[str]:
if self.element_type == ElementType.POINTER:
return [f'TEST_ASSERT(after->{self.name} != NULL);']
return super().check_value()
class CFieldSpecial(CField):
# pylint: disable=too-few-public-methods
"""Field with a custom check."""
custom_behavior: List[str]
def __init__(self, name: str, conditional: List[str], element_type: ElementType,
custom_behavior: List[str]):
self.custom_behavior = custom_behavior
super().__init__(name, conditional, element_type)
def check_value(self) -> List[str]:
return self.custom_behavior
class CStruct:
# pylint: disable=too-few-public-methods
"""Information about the fields of a C struct."""
_PREPROCESSOR_RE = re.compile(r'\s*#\s*(\w+)\s*(.*)')
_STRUCT_RE = re.compile(r'struct\s+(\w+)\s*{')
_FIELD_RE = re.compile(r'\s*([^;]+);')
_PRIVATE_FIELD_RE = re.compile(r'MBEDTLS_PRIVATE\((\w+)\)')
_BARE_FIELD_RE = re.compile(r'[\t *](\w+)\Z')
_NON_BLANK_RE = re.compile(r'.*\S')
def _get_element_type(self, name: str, declaration: str) -> ElementType:
"""Return structure field type based on either its name or the fact
that it belongs to the list of special symbols/named structures"""
# Check for fields with custom check rules
if name in self.fields_info.special:
return ElementType.SPECIAL
# Check for named structures
if name in self.fields_info.named_structures:
return ElementType.NAMED_STRUCTURE
# Check for pointer
if '*' in declaration:
return ElementType.POINTER
# Check for array
if '[' in declaration:
return ElementType.ARRAY
# If we get here then the field is a scalar
return ElementType.SCALAR
def _parse_field(self, declaration: str, conditionals: List[str]) -> CField:
"""Return the CField object describing the given field declaration."""
# Note that this simplistic parsing finds fields in inline
# sub-structs, unions and enums.
m = self._PRIVATE_FIELD_RE.search(declaration)
if not m:
m = self._BARE_FIELD_RE.search(declaration)
if not m:
raise Exception(f'Field name not found in "{declaration}"')
name = m.group(1)
# Get the expected behavior on reset
if name not in self.fields_info.rules:
raise Exception(f'Field {name} does not have an associated behavior')
behavior = self.fields_info.rules[name]
element_type = self._get_element_type(name, declaration)
if behavior == ResetBehavior.SPECIAL:
return CFieldSpecial(name, conditionals, element_type,
self.fields_info.special[name])
elif behavior == ResetBehavior.KEEP:
return CFieldKeep(name, conditionals, element_type)
elif behavior == ResetBehavior.REALLOCATE:
return CFieldReallocate(name, conditionals, element_type)
elif behavior == ResetBehavior.IGNORE:
return CFieldIgnore(name, conditionals, element_type)
elif behavior == ResetBehavior.RESET:
return CFieldReset(name, conditionals, element_type)
else:
raise Exception(f'Unhandled behavior {behavior}')
@staticmethod
def _continue_parsing_preprocessor(arguments: str,
lines: Iterator[Tuple[int, str]]) -> str:
"""Append continuation lines of preprocesssor directive."""
while True:
try:
line_content = next(lines)[1]
arguments = arguments + '\n' + line_content
except StopIteration:
raise Exception('Unexpected end of file reached while '
' parsing a C preprocessor directive')
if not line_content.endswith('\\'):
break
return arguments
def _structure_fields(self,
lines: Iterator[Tuple[int, str]],
struct_name: str) -> Iterator[CField]:
"""Yield a CField object for each field of the given structure."""
# pylint: disable=too-many-branches
found_start = False
for num, line in lines:
m = self._STRUCT_RE.match(line)
if m and m.group(1) == struct_name:
found_start = True
break
if not found_start:
raise Exception(f'Definition of struct {struct_name} not found')
conditionals: List[str] = []
for num, line in lines:
if line.startswith('}'):
return
m = self._PREPROCESSOR_RE.match(line)
if m:
# If the preprocessor directives are included in parentheses (ex:
# "(defined(AAA) && defined(BBB))") then 'line' contains the full directive
# including new lines (if present) so we can just copy that.
# If instead outer parentheses are missing, ex: "defined(AAA) && defined(BBB)"
# then parsing stops at the end of the line so we need to keep parsing
# manually if there is a "\" at the end of the line.
if line.endswith('\\'):
arguments = m.group(2)
arguments = self._continue_parsing_preprocessor(arguments, lines)
one_conditional = '#if ' + arguments
else:
one_conditional = line
one_conditional = one_conditional + '\n'
directive = m.group(1)
if directive == 'if':
conditionals.append(one_conditional)
elif directive == 'endif':
del conditionals[-1]
else:
raise Exception(f'Unsupported directive #{directive} at line {num}')
continue
m = self._FIELD_RE.match(line)
if m:
yield self._parse_field(m.group(1), conditionals)
continue
m = self._NON_BLANK_RE.match(line)
if m:
raise Exception(f'Failed to parse non-empty line {num}. Content is: {line}')
raise Exception(f'End of definition of struct {struct_name} not found')
def _check_special_fields(self):
"""Ensure that all the entries FieldsInfo.rules that are given a SPECIAL
behavior also have the corresponding entry in FieldsInfo.special
(and viceversa)"""
in_rules = frozenset(name for name in self.fields_info.rules
if self.fields_info.rules[name] == ResetBehavior.SPECIAL)
in_special = frozenset(self.fields_info.special)
if in_rules != in_special:
raise Exception(f'Fields with SPECIAL rule in FieldsInfo.rules but '
f'not listed in FieldsInfo.special: {in_rules - in_special}. '
f'Fields listed FieldsInfo.special, but not given a'
f'SPECIAL rule in FieldsInfo.rules: {in_special - in_rules}.')
def _check_rules_struct_fields_matching(self):
"""Ensure that for each field of the given FieldsInfo.rules there is
an entry in the parsed C structure and viceversa"""
given = frozenset(name for name in self.fields_info.rules)
parsed = frozenset(field.name for field in self.fields)
if given != parsed:
raise Exception(f'Fields found in the C struct but not given a '
f'reset behavior: {parsed - given}. '
f'Fields given a reset behavior but not found in '
f'the C struct: {given - parsed}')
def __init__(self, file_name: str, struct_name: str,
fields_info: FieldsInfo) -> None:
"""Parse a structure definition in a C source file."""
self.fields_info = fields_info
self._check_special_fields()
lines = c_parsing_helper.read_logical_lines(file_name)
self.fields = list(self._structure_fields(lines, struct_name))
self._check_rules_struct_fields_matching()
class SSLContextStruct(CStruct):
# pylint: disable=too-few-public-methods
"""Information about the fields of struct mbedtls_ssl_context."""
def __init__(self, fields_info: FieldsInfo) -> None:
super().__init__('include/mbedtls/ssl.h', 'mbedtls_ssl_context',
fields_info)
def write_check_function(self, out: typing_util.Writable) -> None:
"""Write the generated context-checking function to the output."""
out.write(f"""\
/*
* Copyright The Mbed TLS Contributors
* SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
*/
/*
* The following function was automatically generated through the script
* {sys.argv[0]}.
*/
#include <test/ssl_helpers.h>
#include <test/ssl_helpers_internal.h>
#include "mbedtls/psa_util.h"
#include <test/macros.h>
#include <limits.h>
#if defined(MBEDTLS_SSL_TLS_C)
int mbedtls_test_ssl_check_context_after_session_reset(const mbedtls_ssl_context *before,
const mbedtls_ssl_context *after)
{{
mbedtls_ssl_context initial;
int ret = -1;
/* Create a freshly initialized SSL context*/
memset(&initial, 0, sizeof(initial));
mbedtls_ssl_init(&initial);
TEST_EQUAL(mbedtls_ssl_setup(&initial, after->conf), 0);
/* *INDENT-OFF* */
""")
for field in self.fields:
if len(field.conditionals) > 0:
out.write('\n'.join(field.conditionals))
for check_line in field.check_value():
out.write(f' {check_line}\n')
if len(field.conditionals) > 0:
out.write('#endif\n' * len(field.conditionals))
out.write(f"""\
/* *INDENT-ON* */
ret = 0;
exit:
mbedtls_ssl_free(&initial);
return ret;
}}
#endif /* MBEDTLS_SSL_TLS_C */
""")
def main(fields_info: FieldsInfo):
if not build_tree.looks_like_mbedtls_root(os.curdir):
raise Exception("The script must be launched from the root path of Mbed TLS")
arg_parser = argparse.ArgumentParser()
arg_parser.add_argument('-o', dest='output_file',
help='Generated output file',
default='tests/src/ssl_context_reset_verifier.c')
parsed_args = arg_parser.parse_args()
output_file = parsed_args.output_file
with open(output_file, 'wt') as out:
ssl_context = SSLContextStruct(fields_info)
ssl_context.write_check_function(out)

View File

@@ -0,0 +1,175 @@
"""Library for constructing an Mbed TLS test case.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import binascii
import os
import sys
from typing import Iterable, List, Optional
from enum import Enum
from . import build_tree
from . import psa_information
from . import typing_util
HASHES_3_6 = {
"PSA_ALG_MD5" : "MBEDTLS_MD_CAN_MD5",
"PSA_ALG_RIPEMD160" : "MBEDTLS_MD_CAN_RIPEMD160",
"PSA_ALG_SHA_1" : "MBEDTLS_MD_CAN_SHA1",
"PSA_ALG_SHA_224" : "MBEDTLS_MD_CAN_SHA224",
"PSA_ALG_SHA_256" : "MBEDTLS_MD_CAN_SHA256",
"PSA_ALG_SHA_384" : "MBEDTLS_MD_CAN_SHA384",
"PSA_ALG_SHA_512" : "MBEDTLS_MD_CAN_SHA512",
"PSA_ALG_SHA3_224" : "MBEDTLS_MD_CAN_SHA3_224",
"PSA_ALG_SHA3_256" : "MBEDTLS_MD_CAN_SHA3_256",
"PSA_ALG_SHA3_384" : "MBEDTLS_MD_CAN_SHA3_384",
"PSA_ALG_SHA3_512" : "MBEDTLS_MD_CAN_SHA3_512"
}
PK_MACROS_3_6 = {
"PSA_KEY_TYPE_ECC_PUBLIC_KEY" : "MBEDTLS_PK_HAVE_ECC_KEYS"
}
class Domain36(Enum):
PSA = 1
TLS_1_3_ONLY = 2
USE_PSA = 3
LEGACY = 4
def hex_string(data: bytes) -> str:
return '"' + binascii.hexlify(data).decode('ascii') + '"'
class MissingDescription(Exception):
pass
class MissingFunction(Exception):
pass
class TestCase:
"""An Mbed TLS test case."""
def __init__(self, description: Optional[str] = None):
self.comments = [] #type: List[str]
self.description = description #type: Optional[str]
self.dependencies = [] #type: List[str]
self.function = None #type: Optional[str]
self.arguments = [] #type: List[str]
self.skip_reasons = [] #type: List[str]
def add_comment(self, *lines: str) -> None:
self.comments += lines
def set_description(self, description: str) -> None:
self.description = description
def get_dependencies(self) -> List[str]:
return self.dependencies
def set_dependencies(self, dependencies: List[str]) -> None:
self.dependencies = dependencies
def set_function(self, function: str) -> None:
self.function = function
def set_arguments(self, arguments: List[str]) -> None:
self.arguments = arguments
def skip_because(self, reason: str) -> None:
"""Skip this test case.
It will be included in the output, but commented out.
This is intended for test cases that are obtained from a
systematic enumeration, but that have dependencies that cannot
be fulfilled. Since we don't want to have test cases that are
never executed, we arrange not to have actual test cases. But
we do include comments to make it easier to understand the output
of test case generation.
reason must be a non-empty string explaining to humans why this
test case is skipped.
"""
self.skip_reasons.append(reason)
def check_completeness(self) -> None:
if self.description is None:
raise MissingDescription
if self.function is None:
raise MissingFunction
def write(self, out: typing_util.Writable) -> None:
"""Write the .data file paragraph for this test case.
The output starts and ends with a single newline character. If the
surrounding code writes lines (consisting of non-newline characters
and a final newline), you will end up with a blank line before, but
not after the test case.
"""
self.check_completeness()
assert self.description is not None # guide mypy
assert self.function is not None # guide mypy
out.write('\n')
for line in self.comments:
out.write('# ' + line + '\n')
prefix = ''
if self.skip_reasons:
prefix = '## '
for reason in self.skip_reasons:
out.write('## # skipped because: ' + reason + '\n')
out.write(prefix + self.description + '\n')
dependencies = self.get_dependencies()
if dependencies:
out.write(prefix + 'depends_on:' +
':'.join(dependencies) + '\n')
out.write(prefix + self.function + ':' +
':'.join(self.arguments) + '\n')
def write_data_stream(out,
test_cases: Iterable[TestCase],
caller: Optional[str] = None) -> None:
"""Write the test cases to the specified output stream."""
if caller is None:
caller = os.path.basename(sys.argv[0])
out.write('# Automatically generated by {}. Do not edit!\n'
.format(caller))
for tc in test_cases:
tc.write(out)
out.write('\n# End of automatically generated file.\n')
def write_data_file(filename: str,
test_cases: Iterable[TestCase],
caller: Optional[str] = None) -> None:
"""Write the test cases to the specified file.
If the file already exists, it is overwritten.
"""
tempfile = filename + '.new'
with open(tempfile, 'w') as out:
write_data_stream(out, test_cases, caller)
os.replace(tempfile, filename)
def psa_or_3_6_feature_macro(psa_name: str,
domain_3_6: Domain36) -> str:
"""Determine the dependency symbol for a given psa_name based on
the domain and Mbed TLS version. For more information about the domains,
and MBEDTLS_MD_CAN_ prefixed symbols, see transition-guards.md.
This function currently works with hashes and some PK symbols only.
It accepts PSA_ALG_xxx or PSA_KEY_TYPE_xxx as inputs for psa_name.
"""
if domain_3_6 == Domain36.PSA or domain_3_6 == Domain36.TLS_1_3_ONLY or \
not build_tree.is_mbedtls_3_6():
if psa_name in PK_MACROS_3_6 or psa_name in HASHES_3_6:
return psa_information.psa_want_symbol(psa_name)
if domain_3_6 == Domain36.USE_PSA:
if psa_name in PK_MACROS_3_6:
return PK_MACROS_3_6[psa_name]
if psa_name in HASHES_3_6:
return HASHES_3_6[psa_name]
raise ValueError(f'Unable to determine dependency symbol for {psa_name} in {domain_3_6}')

View File

@@ -0,0 +1,249 @@
"""Common code for test data generation.
This module defines classes that are of general use to automatically
generate .data files for unit tests, as well as a main function.
These are used both by generate_psa_tests.py and generate_bignum_tests.py.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import argparse
import io
import os
import posixpath
import re
import inspect
from abc import ABCMeta, abstractmethod
from typing import Callable, Dict, Iterable, Iterator, List, Type, TypeVar
from . import build_tree
from . import test_case
T = TypeVar('T') #pylint: disable=invalid-name
class BaseTest(metaclass=ABCMeta):
"""Base class for test case generation.
Attributes:
count: Counter for test cases from this class.
case_description: Short description of the test case. This may be
automatically generated using the class, or manually set.
dependencies: A list of dependencies required for the test case.
show_test_count: Toggle for inclusion of `count` in the test description.
test_function: Test function which the class generates cases for.
test_name: A common name or description of the test function. This can
be `test_function`, a clearer equivalent, or a short summary of the
test function's purpose.
"""
count = 0
case_description = ""
dependencies = [] # type: List[str]
show_test_count = True
test_function = ""
test_name = ""
def __new__(cls, *args, **kwargs):
# pylint: disable=unused-argument
cls.count += 1
return super().__new__(cls)
@abstractmethod
def arguments(self) -> List[str]:
"""Get the list of arguments for the test case.
Override this method to provide the list of arguments required for
the `test_function`.
Returns:
List of arguments required for the test function.
"""
raise NotImplementedError
def description(self) -> str:
"""Create a test case description.
Creates a description of the test case, including a name for the test
function, an optional case count, and a description of the specific
test case. This should inform a reader what is being tested, and
provide context for the test case.
Returns:
Description for the test case.
"""
if self.show_test_count:
return "{} #{} {}".format(
self.test_name, self.count, self.case_description
).strip()
else:
return "{} {}".format(self.test_name, self.case_description).strip()
def create_test_case(self) -> test_case.TestCase:
"""Generate TestCase from the instance."""
tc = test_case.TestCase()
tc.set_description(self.description())
tc.set_function(self.test_function)
tc.set_arguments(self.arguments())
tc.set_dependencies(self.dependencies)
return tc
@classmethod
@abstractmethod
def generate_function_tests(cls) -> Iterator[test_case.TestCase]:
"""Generate test cases for the class test function.
This will be called in classes where `test_function` is set.
Implementations should yield TestCase objects, by creating instances
of the class with appropriate input data, and then calling
`create_test_case()` on each.
"""
raise NotImplementedError
class BaseTarget:
#pylint: disable=too-few-public-methods
"""Base target for test case generation.
Child classes of this class represent an output file, and can be referred
to as file targets. These indicate where test cases will be written to for
all subclasses of the file target, which is set by `target_basename`.
Attributes:
target_basename: Basename of file to write generated tests to. This
should be specified in a child class of BaseTarget.
"""
target_basename = ""
@classmethod
def generate_tests(cls) -> Iterator[test_case.TestCase]:
"""Generate test cases for the class and its subclasses.
In classes with `test_function` set, `generate_function_tests()` is
called to generate test cases first.
In all classes, this method will iterate over its subclasses, and
yield from `generate_tests()` in each. Calling this method on a class X
will yield test cases from all classes derived from X.
"""
if issubclass(cls, BaseTest) and not inspect.isabstract(cls):
#pylint: disable=no-member
yield from cls.generate_function_tests()
for subclass in sorted(cls.__subclasses__(), key=lambda c: c.__name__):
yield from subclass.generate_tests()
class TestGenerator:
"""Generate test cases and write to data files."""
# Note that targets whose names contain 'test_format' have their content
# validated by `abi_check.py`.
targets = {} # type: Dict[str, Callable[..., Iterable[test_case.TestCase]]]
def __init__(self, options) -> None:
self.test_suite_directory = options.directory
# Update `targets` with an entry for each child class of BaseTarget.
# Each entry represents a file generated by the BaseTarget framework,
# and enables generating the .data files using the CLI.
self.targets.update({
subclass.target_basename: subclass.generate_tests
for subclass in BaseTarget.__subclasses__()
if subclass.target_basename
})
def filename_for(self, basename: str) -> str:
"""The location of the data file with the specified base name."""
return posixpath.join(self.test_suite_directory, basename + '.data')
def write_test_data_file(self, basename: str,
test_cases: Iterable[test_case.TestCase]) -> None:
"""Write the test cases to a .data file.
The output file is ``basename + '.data'`` in the test suite directory.
"""
filename = self.filename_for(basename)
test_case.write_data_file(filename, test_cases)
def generate_target(self, name: str, *target_args) -> None:
"""Generate cases and write to data file for a target.
For target callables which require arguments, override this function
and pass these arguments using super() (see PSATestGenerator).
"""
test_cases = self.targets[name](*target_args)
self.write_test_data_file(name, test_cases)
def is_up_to_date(self, target) -> bool:
"""Check if the given target already has the expected content."""
filename = self.filename_for(target)
if not os.path.exists(filename):
return False
test_cases = self.targets[target]()
out = io.StringIO()
test_case.write_data_stream(out, test_cases)
out.seek(0)
new_content = out.read()
out.close()
with open(filename) as current_file:
old_content = current_file.read()
return new_content == old_content
def main(args, description: str, generator_class: Type[TestGenerator] = TestGenerator):
"""Command line entry point."""
parser = argparse.ArgumentParser(description=description)
parser.add_argument('--list', action='store_true',
help='List available targets and exit')
parser.add_argument('--list-for-cmake', action='store_true',
help='Print \';\'-separated list of available targets and exit')
parser.add_argument('--list-outdated', action='store_true',
help=('List outdated targets and exit '
'(succeeds even if there are outdated or missing targets)'))
# If specified explicitly, this option may be a path relative to the
# current directory when the script is invoked. The default value
# is relative to the mbedtls root, which we don't know yet. So we
# can't set a string as the default value here.
parser.add_argument('--directory', metavar='DIR',
help='Output directory (default: tests/suites)')
parser.add_argument('targets', nargs='*', metavar='TARGET',
help='Target file to generate (default: all; "-": none)')
options = parser.parse_args(args)
# Change to the mbedtls root, to keep things simple. But first, adjust
# command line options that might be relative paths.
if options.directory is None:
options.directory = 'tests/suites'
else:
options.directory = os.path.abspath(options.directory)
build_tree.chdir_to_root()
generator = generator_class(options)
if options.list:
for name in sorted(generator.targets):
print(generator.filename_for(name))
return
# List in a cmake list format (i.e. ';'-separated)
if options.list_for_cmake:
print(';'.join(generator.filename_for(name)
for name in sorted(generator.targets)), end='')
return
if options.targets:
# Allow "-" as a special case so you can run
# ``generate_xxx_tests.py - $targets`` and it works uniformly whether
# ``$targets`` is empty or not.
options.targets = [os.path.basename(re.sub(r'\.data\Z', r'', target))
for target in options.targets
if target != '-']
else:
options.targets = sorted(generator.targets)
for target in options.targets:
if options.list_outdated:
if not generator.is_up_to_date(target):
print(generator.filename_for(target))
else:
generator.generate_target(target)

View File

@@ -0,0 +1,341 @@
"""Library for building a TF-PSA-Crypto test driver from the built-in driver
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import argparse
import re
import shutil
import subprocess
from fnmatch import fnmatch
from pathlib import Path
from typing import Iterable, List, Match, Optional, Set
def get_parsearg_base() -> argparse.ArgumentParser:
""" Get base arguments for scripts generating a TF-PSA-Crypto test driver """
parser = argparse.ArgumentParser(description="""\
Clone the built-in driver tree, rewrite header inclusions and prefix
exposed C identifiers.
""")
parser.add_argument("dst_dir", metavar="DST_DIR",
help="Destination directory (relative to repository root)")
parser.add_argument("--driver", default="libtestdriver1", metavar="DRIVER",
help="Test driver name (default: %(default)s).")
parser.add_argument('--list-vars-for-cmake', nargs="?", \
const="__AUTO__", metavar="FILE",
help="""
Generate a file to be included from a CMakeLists.txt and exit. The file
defines CMake list variables with the script's inputs/outputs files. If
FILE is omitted, the output name defaults to '<DRIVER>-list-vars.cmake'.
""")
return parser
class TestDriverGenerator:
"""A TF-PSA-Crypto test driver generator"""
def __init__(self, src_dir: Path, dst_dir: Path, driver: str, \
exclude_files: Optional[Iterable[str]] = None) -> None:
"""
Initialize a test driver generator.
Args:
src_dir (Path):
Path to the source directory that contains the built-in driver.
If this path is relative, it should be relative to the repository
root so that the source paths returned by `write_list_vars_for_cmake`
are correct.
The source directory is expected to contain:
- an `include` directory
- an `src` directory
dst_dir (Path):
Path to the destination directory where the rewritten tree will
be created. If the directory already exists, only the `include`
and `src` subdirectories are modified.
driver (str):
Name of the driver. This is used as a prefix when rewritting
the tree.
exclude_files (Optional[Iterable[str]]):
Glob patterns for the basename of the files to be excluded from
the source directory.
"""
self.src_dir = src_dir
self.dst_dir = dst_dir
self.driver = driver
self.exclude_files = [] if exclude_files is None else list(exclude_files)
if not (src_dir / "include").is_dir():
raise RuntimeError(f'"include" directory in {src_dir} not found')
if not (src_dir / "src").is_dir():
raise RuntimeError(f'"src" directory in {src_dir} not found')
def write_list_vars_for_cmake(self, fname: str) -> None:
src_relpaths = self.__get_src_code_files()
with open(self.dst_dir / fname, "w") as f:
f.write(f"set({self.driver}_input_files\n " + \
"\n ".join(f'"{path}"' for path in src_relpaths) + "\n)\n\n")
f.write(f"set({self.driver}_files\n " + \
"\n ".join(f'"{self.__get_dst_relpath(path.relative_to(self.src_dir))}"' \
for path in src_relpaths) + "\n)\n\n")
f.write(f"set({self.driver}_src_files\n " + \
"\n ".join(f'"{self.__get_dst_relpath(path.relative_to(self.src_dir))}"' \
for path in src_relpaths if path.suffix == ".c") + "\n)\n")
def get_identifiers_to_prefix(self, prefixes: Set[str]) -> Set[str]:
"""
Get the set of identifiers that will be prefixed in the test driver code.
This method is intended to be amended by subclasses in consuming branches.
The default implementation returns the complete set of identifiers from
the built-in driver whose names begin with any of the `prefixes`. These
are the identifiers that could be renamed in the test driver before
adaptation.
Subclasses need to filter, transform, or otherwise adjust the set of
identifiers that should be renamed when generating the test driver.
Args:
prefixes (Set[str]):
The set of identifier prefixes used by the built-in driver
for the symbols it exposes to the other parts of the crypto
library. All identifiers beginning with any of these
prefixes are candidates for renaming in the test driver to
avoid symbol clashes.
Returns:
Set[str]: The default set of identifiers to rename.
"""
identifiers = set()
for file in self.__get_src_code_files():
identifiers.update(self.get_c_identifiers(file))
identifiers_with_prefixes = set()
for identifier in identifiers:
if any(identifier.startswith(prefix) for prefix in prefixes):
identifiers_with_prefixes.add(identifier)
return identifiers_with_prefixes
def create_test_driver_tree(self, prefixes: Set[str]) -> None:
"""
Create a test driver tree from `self.src_dir` into `self.dst_dir`.
Only the `include/` and `src/` subdirectories of the source tree are
used, and their internal directory structure is preserved.
Only "*.h" and "*.c" files are copied. Files whose basenames match any
of the glob patterns in `self.exclude_files` are excluded.
The basename of all files is prefixed with `{self.driver}-`. The
header inclusions referencing the renamed headers are rewritten
accordingly.
Symbol identifiers exposed by the built-in driver are renamed by
prefixing them with `{self.driver}_` to avoid collisions when linking the
built-in driver and the test driver together in the crypto library.
Args:
prefixes (Set[str]):
The set of identifier prefixes used by the built-in driver
for the symbols it exposes to the other parts of the crypto
library. All identifiers beginning with any of these
prefixes are candidates for renaming in the test driver to
avoid symbol clashes.
"""
if (self.dst_dir / "include").exists():
shutil.rmtree(self.dst_dir / "include")
if (self.dst_dir / "src").exists():
shutil.rmtree(self.dst_dir / "src")
headers = {
f.name \
for f in self.__get_src_code_files() if f.suffix == ".h"
}
identifiers_to_prefix = self.get_identifiers_to_prefix(prefixes)
# Create the test driver tree
for file in self.__get_src_code_files():
dst = self.dst_dir / \
self.__get_dst_relpath(file.relative_to(self.src_dir))
dst.parent.mkdir(parents=True, exist_ok=True)
self.__write_test_driver_file(file, dst, headers,\
identifiers_to_prefix)
@staticmethod
def __get_code_files(root: Path) -> List[Path]:
"""
Return all "*.c" and "*.h" files found recursively under the
`include` and `src` subdirectories of `root`.
"""
return sorted(path
for directory in ('include', 'src')
for path in (root / directory).rglob('*.[hc]'))
def __get_src_code_files(self) -> List[Path]:
"""
Return all "*.c" and "*.h" files found recursively under the
`include` and `src` subdirectories of the source directory `self.src_dir`
excluding the files whose basename match any of the patterns in
`self.exclude_files`.
"""
out = []
for file in self.__get_code_files(self.src_dir):
if not any(fnmatch(file.name, pattern) for pattern in self.exclude_files):
out.append(file)
return out
def __get_dst_relpath(self, src_relpath: Path) -> Path:
"""
Return the path relative to `dst_dir` of the file that corresponds to the
file with relative path `src_relpath` in the source tree.
Same as `src_relpath` but the basename prefixed with `self.driver`
"""
assert not src_relpath.is_absolute(), "src_relpath must be relative"
return src_relpath.parent / (self.driver + '-' + src_relpath.name)
@staticmethod
def get_c_identifiers(file: Path) -> Set[str]:
"""
Extract the C identifiers present in `file` using `ctags -x`
The following C symbol kinds are included (with their `--c-kinds`
flags in parentheses):
- macro definitions (d)
- enum values (e)
- functions (f)
- enum tags (g)
- function prototypes (p)
- struct tags (s)
- typedefs (t)
- union tags (u)
- global variables (v)
Compatibility
-------------
The command used here has been validated with the following `ctags`
implementations:
- Exuberant Ctags 5.8
- Exuberant Ctags 5.9~svn20110310 (default on Ubuntu 16.0424.04)
- Universal Ctags 5.9.0 (Ubuntu 24.04)
- Universal Ctags 6.2.0 (Ubuntu 26.04)
All of these versions support the options `-x`, `--language-force=C`,
and ``--c-kinds=defgpstuv`` sufficiently for the use case here.
Returns:
Set[str]: The set of identifiers found in `file`.
"""
output = subprocess.check_output(
["ctags", "-x", "--language-force=C", "--c-kinds=defgpstuv", str(file)],
stderr=subprocess.STDOUT,
universal_newlines=True,
)
identifiers = set()
for line in output.splitlines():
identifiers.add(line.split()[0])
return identifiers
def __write_test_driver_file(self, src: Path, dst: Path,
headers: Set[str],
identifiers_to_prefix: Set[str]) -> None:
"""
Write a test driver file to `dst` based on the contents of `src` with
two transformations: rewriting of `#include` directives and identifier
renaming.
1. Rewrite header inclusions
Any `#include` directive whose header basename matches an entry of
`headers` is rewritten so that the basename is prefixed with
`{self.driver}-`. Directory components (if any) are preserved.
Example:
#include "mbedtls/private/aes.h"
becomes
#include "mbedtls/private/libtestdriver1-aes.h"
LIMITATION:
The current implementation does not correctly handle the case
where a built-in header and a nonbuilt-in header share the same
basename. In principle, only inclusions of built-in headers
should be rewritten, while inclusions of nonbuilt-in headers
should be left unchanged. However, the current logic only matches
on the basename, so both are rewritten.
For example, if both `include/psa/foo.h` (nonbuilt-in) and
`drivers/builtin/include/mbedtls/foo.h` (built-in) exist, then
in the test driver:
- `#include <psa/foo.h>` should not be rewritten
- `#include <mbedtls/foo.h>` should be rewritten to
`#include <mbedtls/libtestdriver1-foo.h>`
With the current basename-only matching, both inclusions are
rewritten, which is incorrect. No practical implications
currently, such same header basename case does not occur in the
code base.
2. Rename selected identifiers
Each identifier in `identifiers_to_prefix` is prefixed with
`self.driver`. Case is preserved: if the identifier is all-uppercase,
then the uppercase form of `driver` is used, the lowercase form
otherwise.
Examples:
`MBEDTLS_AES_C` becomes `LIBTESTDRIVER1_MBEDTLS_AES_C`
`mbedtls_sha256_init` becomes `libtestdriver1_mbedtls_sha256_init`
Args:
src (Path):
The source file to read.
dst (Path):
The destination file where the rewritten version is written.
headers (Set[str]):
Basenames of headers whose includes should be rewritten.
identifiers_to_prefix (Set[str]):
Identifiers that must be renamed by prefixing with `self.driver`
(using uppercase or lowercase depending on the identifier's casing).
"""
text = src.read_text(encoding="utf-8")
include_line_re = re.compile(
fr'^(\s*#\s*include\s*[<"])([^>"]+)([>"])',
re.MULTILINE
)
def repl_header_inclusion(m: Match) -> str:
parts = m.group(2).split("/")
if parts[-1] in headers:
path = "/".join(parts[:-1] + [self.driver + "-" + parts[-1]])
return f'{m.group(1)}{path}{m.group(3)}'
return m.group(0)
intermediate_text = include_line_re.sub(repl_header_inclusion, text)
c_identifier_re = re.compile(r"\b[A-Za-z_][A-Za-z0-9_]*\b")
prefix_uppercased = self.driver.upper()
prefix_lowercased = self.driver.lower()
def repl(m: Match) -> str:
identifier = m.group(0)
if identifier in identifiers_to_prefix:
if identifier[0].isupper():
return f"{prefix_uppercased}_{identifier}"
else:
return f"{prefix_lowercased}_{identifier}"
return identifier
new_text = c_identifier_re.sub(repl, intermediate_text)
dst.write_text(new_text, encoding="utf-8")

View File

@@ -0,0 +1,228 @@
"""
Generate miscellaneous TLS test cases relating to the handshake.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
import os
import sys
from typing import Optional
from mbedtls_framework import tls_test_case
from mbedtls_framework import typing_util
from mbedtls_framework.tls_test_case import Side, Version
import translate_ciphers
# Assume that a TLS 1.2 ClientHello used in these tests will be at most
# this many bytes long.
TLS12_CLIENT_HELLO_ASSUMED_MAX_LENGTH = 255
# Minimum handshake fragment length that Mbed TLS supports.
TLS_HANDSHAKE_FRAGMENT_MIN_LENGTH = 4
def write_tls_handshake_defragmentation_test(
#pylint: disable=too-many-arguments
out: typing_util.Writable,
side: Side,
length: Optional[int],
version: Optional[Version] = None,
cipher: Optional[str] = None,
etm: Optional[bool] = None, #encrypt-then-mac (only relevant for CBC)
tls12_client_hello_defragmentation: Optional[bool] = True,
variant: str = ''
) -> None:
"""Generate one TLS handshake defragmentation test.
:param out: file to write to.
:param side: which side is Mbed TLS.
:param length: fragment length, or None to not fragment.
:param version: protocol version, if forced.
"""
#pylint: disable=chained-comparison,too-many-branches,too-many-statements
our_args = ''
their_args = ''
if length is None:
description = 'no fragmentation, for reference'
else:
description = 'len=' + str(length)
if version is not None:
description += ', TLS 1.' + str(version.value)
description = f'Handshake defragmentation on {side.name.lower()}: {description}'
tc = tls_test_case.TestCase(description)
if version is not None:
their_args += ' ' + version.openssl_option()
# Emit a version requirement, because we're forcing the version via
# OpenSSL, not via Mbed TLS, and the automatic depdendencies in
# ssl-opt.sh only handle forcing the version via Mbed TLS.
tc.requirements.append(version.requires_command())
if side == Side.SERVER and version == Version.TLS12 and \
length is not None and \
length <= TLS12_CLIENT_HELLO_ASSUMED_MAX_LENGTH and \
not tls12_client_hello_defragmentation:
# If Server-side ClientHello defragmentation is only supported in
# the TLS 1.3 message parser, not in the TLS 1.2 message parser,
# a TLS 1.2 fragmented ClientHello is handled properly only if it
# is first reassembled by the TLS 1.3 parser before to be passed to
# the TLS 1.2 ClientHello parser in a TLS 1.3 or TLS 1.2 version
# negotiation scenario.
# When TLS 1.3 support is disabled in the server (at compile-time
# or at runtime), the TLS 1.2 ClientHello parser only sees
# the first fragment of a fragmented ClientHello.
tc.requirements.append('requires_config_enabled MBEDTLS_SSL_PROTO_TLS1_3')
tc.description += ' with 1.3 support'
# To guarantee that the handhake messages are large enough and need to be
# split into fragments, the tests require certificate authentication.
# The party in control of the fragmentation operations is OpenSSL and
# will always use server5.crt (548 Bytes).
if length is not None and \
length >= TLS_HANDSHAKE_FRAGMENT_MIN_LENGTH:
tc.requirements.append('requires_certificate_authentication')
if version == Version.TLS12 and side == Side.CLIENT:
#The server uses an ECDSA cert, so make sure we have a compatible key exchange
tc.requirements.append(
'requires_config_enabled MBEDTLS_KEY_EXCHANGE_ECDHE_ECDSA_ENABLED')
else:
# This test case may run in a pure-PSK configuration. OpenSSL doesn't
# allow this by default with TLS 1.3.
their_args += ' -allow_no_dhe_kex'
if length is None:
forbidden_patterns = [
'waiting for more fragments',
]
wanted_patterns = []
elif length < TLS_HANDSHAKE_FRAGMENT_MIN_LENGTH:
their_args += ' -split_send_frag ' + str(length)
tc.exit_code = 1
forbidden_patterns = []
wanted_patterns = [
'handshake message too short: ' + str(length),
'SSL - An invalid SSL record was received',
]
if side == Side.SERVER:
wanted_patterns[0:0] = ['=> parse client hello']
elif version == Version.TLS13:
wanted_patterns[0:0] = ['=> ssl_tls13_process_server_hello']
else:
their_args += ' -split_send_frag ' + str(length)
forbidden_patterns = []
wanted_patterns = [
'reassembled record',
fr'initial handshake fragment: {length}, 0\.\.{length} of [0-9]\+',
fr'subsequent handshake fragment: [0-9]\+, {length}\.\.',
fr'Prepare: waiting for more handshake fragments {length}/',
fr'Consume: waiting for more handshake fragments {length}/',
]
if cipher is not None:
mbedtls_cipher = translate_ciphers.translate_mbedtls(cipher)
if side == Side.CLIENT:
our_args += ' force_ciphersuite=' + mbedtls_cipher
if 'NULL' in cipher:
their_args += ' -cipher ALL@SECLEVEL=0:COMPLEMENTOFALL@SECLEVEL=0'
else:
# For TLS 1.2, when Mbed TLS is the server, we must force the
# cipher suite on the client side, because passing
# force_ciphersuite to ssl_server2 would force a TLS-1.2-only
# server, which does not support a fragmented ClientHello.
tc.requirements.append('requires_ciphersuite_enabled ' + mbedtls_cipher)
their_args += ' -cipher ' + translate_ciphers.translate_ossl(cipher)
if 'NULL' in cipher:
their_args += '@SECLEVEL=0'
if etm is not None:
if etm:
tc.requirements.append('requires_config_enabled MBEDTLS_SSL_ENCRYPT_THEN_MAC')
our_args += ' etm=' + str(int(etm))
(wanted_patterns if etm else forbidden_patterns)[0:0] = [
'using encrypt then mac',
]
tc.description += variant
if side == Side.CLIENT:
tc.client = '$P_CLI debug_level=4' + our_args
tc.server = '$O_NEXT_SRV' + their_args
tc.wanted_client_patterns = wanted_patterns
tc.forbidden_client_patterns = forbidden_patterns
else:
their_args += ' -cert $DATA_FILES_PATH/server5.crt -key $DATA_FILES_PATH/server5.key'
our_args += ' auth_mode=required'
tc.client = '$O_NEXT_CLI' + their_args
tc.server = '$P_SRV debug_level=4' + our_args
tc.wanted_server_patterns = wanted_patterns
tc.forbidden_server_patterns = forbidden_patterns
tc.write(out)
CIPHERS_FOR_TLS12_HANDSHAKE_DEFRAGMENTATION = [
(None, 'default', None),
('TLS_ECDHE_ECDSA_WITH_NULL_SHA', 'null', None),
('TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256', 'ChachaPoly', None),
('TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256', 'GCM', None),
('TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256', 'CBC, etm=n', False),
('TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256', 'CBC, etm=y', True),
]
def write_tls_handshake_defragmentation_tests(args, out: typing_util.Writable) -> None:
"""Generate TLS handshake defragmentation tests."""
for side in Side.CLIENT, Side.SERVER:
write_tls_handshake_defragmentation_test(out, side, None)
for length in [512, 513, 256, 128, 64, 36, 32, 16, 13, 5, 4, 3]:
write_tls_handshake_defragmentation_test(out, side, length,
Version.TLS13)
if length == 4:
for (cipher_suite, nickname, etm) in \
CIPHERS_FOR_TLS12_HANDSHAKE_DEFRAGMENTATION:
write_tls_handshake_defragmentation_test(
out, side, length, Version.TLS12,
cipher=cipher_suite, etm=etm,
variant=', '+nickname,
tls12_client_hello_defragmentation= \
args.tls12_client_hello_defragmentation)
else:
write_tls_handshake_defragmentation_test(
out, side, length, Version.TLS12,
tls12_client_hello_defragmentation=
args.tls12_client_hello_defragmentation)
def write_handshake_tests(args, out: typing_util.Writable) -> None:
"""Generate handshake tests."""
out.write(f"""\
# Miscellaneous tests related to the TLS handshake layer.
#
# Automatically generated by {os.path.basename(sys.argv[0])}. Do not edit!
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
""")
write_tls_handshake_defragmentation_tests(args, out)
out.write("""\
# End of automatically generated file.
""")
def main() -> None:
"""Command line entry point."""
parser = argparse.ArgumentParser()
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('-o', '--output',
default='tests/opt-testcases/handshake-generated.sh',
help='Output file (default: tests/opt-testcases/handshake-generated.sh)')
parser.add_argument('--no-tls12-client-hello-defragmentation-support',
action="store_false",
dest="tls12_client_hello_defragmentation",
help="Whether the TLS 1.2 ClientHello defragmentation is "
"fully supported or not (default: True)")
args = parser.parse_args()
with open(args.output, 'w') as out:
write_handshake_tests(args, out)

View File

@@ -0,0 +1,101 @@
"""Library for constructing an Mbed TLS ssl-opt test case.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import enum
import re
from typing import List
from . import typing_util
class TestCase:
"""Data about an ssl-opt test case."""
#pylint: disable=too-few-public-methods
def __init__(self, description: str) -> None:
# List of shell snippets to call before run_test, typically
# calls to requires_xxx functions.
self.requirements = [] #type: List[str]
# Test case description (first argument to run_test).
self.description = description
# Client command line.
# This will be placed directly inside double quotes in the shell script.
self.client = '$P_CLI'
# Server command line.
# This will be placed directly inside double quotes in the shell script.
self.server = '$P_SRV'
# Expected client exit code.
self.exit_code = 0
# Note that all patterns matched in the logs are in BRE
# (Basic Regular Expression) syntax, more precisely in the BRE
# dialect that is the default for GNU grep. The main difference
# with Python regular expressions is that the operators for
# grouping `\(...\)`, alternation `x\|y`, option `x\?`,
# one-or-more `x\+` and repetition ranges `x\{M,N\}` must be
# preceded by a backslash. The characters `()|?+{}` stand for
# themselves.
# BRE for text that must be present in the client log (run_test -c).
self.wanted_client_patterns = [] #type: List[str]
# BRE for text that must be present in the server log (run_test -s).
self.wanted_server_patterns = [] #type: List[str]
# BRE for text that must not be present in the client log (run_test -C).
self.forbidden_client_patterns = [] #type: List[str]
# BRE for text that must not be present in the server log (run_test -S).
self.forbidden_server_patterns = [] #type: List[str]
@staticmethod
def _quote(raw: str) -> str:
"""Quote the given string for sh.
Use double quotes, because that's currently the norm in ssl-opt.sh.
"""
return '"' + re.sub(r'([$"\\`])', r'\\\1', raw) + '"'
def write(self, out: typing_util.Writable) -> None:
"""Write the test case to the specified file."""
for req in self.requirements:
out.write(req + '\n')
out.write(f'run_test {self._quote(self.description)} \\\n')
out.write(f' "{self.server}" \\\n')
out.write(f' "{self.client}" \\\n')
out.write(f' {self.exit_code}')
for pat in self.wanted_server_patterns:
out.write(' \\\n -s ' + self._quote(pat))
for pat in self.forbidden_server_patterns:
out.write(' \\\n -S ' + self._quote(pat))
for pat in self.wanted_client_patterns:
out.write(' \\\n -c ' + self._quote(pat))
for pat in self.forbidden_client_patterns:
out.write(' \\\n -C ' + self._quote(pat))
out.write('\n\n')
class Side(enum.Enum):
CLIENT = 0
SERVER = 1
class Version(enum.Enum):
"""TLS protocol version.
This class doesn't know about DTLS yet.
"""
TLS12 = 2
TLS13 = 3
def force_version(self) -> str:
"""Argument to pass to ssl_client2 or ssl_server2 to force this version."""
return f'force_version=tls1{self.value}'
def openssl_option(self) -> str:
"""Option to pass to openssl s_client or openssl s_server to select this version."""
return f'-tls1_{self.value}'
def requires_command(self) -> str:
"""Command to require this protocol version in an ssl-opt.sh test case."""
return 'requires_config_enabled MBEDTLS_SSL_PROTO_TLS1_' + str(self.value)

View File

@@ -0,0 +1,28 @@
"""Auxiliary definitions used in type annotations.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
from typing import Any
# The typing_extensions module is necessary for type annotations that are
# checked with mypy. It is only used for type annotations or to define
# things that are themselves only used for type annotations. It is not
# available on a default Python installation. Therefore, try loading
# what we need from it for the sake of mypy (which depends on, or comes
# with, typing_extensions), and if not define substitutes that lack the
# static type information but are good enough at runtime.
try:
from typing_extensions import Protocol #pylint: disable=import-error
except ImportError:
class Protocol: #type: ignore
#pylint: disable=too-few-public-methods
pass
class Writable(Protocol):
"""Abstract class for typing hints."""
# pylint: disable=no-self-use,too-few-public-methods,unused-argument
def write(self, text: str) -> Any:
...

View File

@@ -0,0 +1,195 @@
"""Test the configuration checks that reject some bad compile-time configs.
This tests the output of ``generate_config_checks.py``.
This can also let us verify what we enforce in the manually written
checks in ``<PROJECT>_check_config.h`` and ``<PROJECT>_config.c``.
"""
## Copyright The Mbed TLS Contributors
## SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import os
import subprocess
import sys
import unittest
from typing import List, Optional, Pattern, Union
class TestConfigChecks(unittest.TestCase):
"""Unit tests for checks performed via ``<PROJECT>_config.c``.
This can test the code generated by `config_checks_generator`,
as well as manually written checks in `check_config.h`.
"""
# Set this to the path to the source file containing the config checks.
PROJECT_CONFIG_C = None #type: Optional[str]
# Project-specific include directories (in addition to /include)
PROJECT_SPECIFIC_INCLUDE_DIRECTORIES = [] #type: List[str]
# Increase the length of strings that assertion failures are willing to
# print. This is useful for failures where the compiler has a lot
# to say.
maxDiff = 9999
def setUp(self) -> None:
self.cc_output = None #type: Optional[str]
def tearDown(self) -> None:
"""Log the compiler output to a file, if available and desired.
This is intended for debugging. It only happens if the environment
variable UNITTEST_CONFIG_CHECKS_DEBUG is non-empty.
"""
if os.getenv('UNITTEST_CONFIG_CHECKS_DEBUG'):
# We set self.cc_output to the compiler output before
# asserting, and set it to None if all the assertions pass.
if self.cc_output is not None:
basename = os.path.splitext(os.path.basename(sys.argv[0]))[0]
filename = f'{basename}.{self._testMethodName}.out.txt'
with open(filename, 'w') as out:
out.write(self.cc_output)
def user_config_file_name(self, variant: str) -> str:
"""Construct a unique temporary file name for a user config header."""
name = os.path.splitext(os.path.basename(sys.argv[0]))[0]
pid = str(os.getpid())
oid = str(id(self))
return f'tmp-user_config_{variant}-{name}-{pid}-{oid}.h'
def write_user_config(self, variant: str, content: Optional[str]) -> Optional[str]:
"""Write a user configuration file with the given content.
If content is None, ensure the file does not exist.
Return None if content is none, otherwise return the file name.
"""
file_name = self.user_config_file_name(variant)
if content is None:
if os.path.exists(file_name):
os.remove(file_name)
return None
if content and not content.endswith('\n'):
content += '\n'
with open(file_name, 'w', encoding='utf-8') as out:
out.write(content)
return file_name
def run_with_config_files(self,
crypto_user_config_file: Optional[str],
mbedtls_user_config_file: Optional[str],
extra_options: List[str],
) -> subprocess.CompletedProcess:
"""Run cc with the given user configuration files.
Return the CompletedProcess object capturing the return code,
stdout and stderr.
"""
cmd = [os.getenv('CC', 'cc')]
if os.getenv('UNITTEST_CONFIG_CHECKS_DEBUG'):
cmd += ['-dD']
if crypto_user_config_file is not None:
cmd.append(f'-DTF_PSA_CRYPTO_USER_CONFIG_FILE="{crypto_user_config_file}"')
if mbedtls_user_config_file is not None:
cmd.append(f'-DMBEDTLS_USER_CONFIG_FILE="{mbedtls_user_config_file}"')
cmd += extra_options
assert self.PROJECT_CONFIG_C is not None
cmd += ['-I' + dir for dir in self.PROJECT_SPECIFIC_INCLUDE_DIRECTORIES]
cmd += ['-Iinclude',
'-I.',
'-I' + os.path.dirname(self.PROJECT_CONFIG_C)]
cmd += ['-o', os.devnull, '-c', self.PROJECT_CONFIG_C]
return subprocess.run(cmd,
check=False,
encoding='utf-8',
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
def run_with_config(self,
crypto_user_config: Optional[str],
mbedtls_user_config: Optional[str] = None,
extra_options: Optional[List[str]] = None,
) -> subprocess.CompletedProcess:
"""Run cc with the given content for user configuration files.
Return the CompletedProcess object capturing the return code,
stdout and stderr.
"""
if extra_options is None:
extra_options = []
crypto_user_config_file = None
mbedtls_user_config_file = None
try:
# Create temporary files without using tempfile because:
# 1. Before Python 3.12, tempfile.NamedTemporaryFile does
# not have good support for allowing an external program
# to access the file on Windows.
# 2. With a tempfile-provided context, it's awkward to not
# create a file optionally (we only do it when xxx_user_config
# is not None).
crypto_user_config_file = \
self.write_user_config('crypto', crypto_user_config)
mbedtls_user_config_file = \
self.write_user_config('mbedtls', mbedtls_user_config)
cp = self.run_with_config_files(crypto_user_config_file,
mbedtls_user_config_file,
extra_options)
return cp
finally:
if crypto_user_config_file is not None and \
os.path.exists(crypto_user_config_file):
os.remove(crypto_user_config_file)
if mbedtls_user_config_file is not None and \
os.path.exists(mbedtls_user_config_file):
os.remove(mbedtls_user_config_file)
def good_case(self,
crypto_user_config: Optional[str],
mbedtls_user_config: Optional[str] = None,
extra_options: Optional[List[str]] = None,
) -> None:
"""Run cc with the given user config(s). Expect no error.
Pass extra_options on the command line of cc.
"""
cp = self.run_with_config(crypto_user_config, mbedtls_user_config,
extra_options=extra_options)
# Assert the error text before the status. That way, if it fails,
# we see the unexpected error messages in the test log.
self.cc_output = cp.stdout
self.assertEqual(cp.stderr, '')
self.assertEqual(cp.returncode, 0)
self.cc_output = None
def bad_case(self,
crypto_user_config: Optional[str],
mbedtls_user_config: Optional[str] = None,
error: Optional[Union[str, Pattern]] = None,
extra_options: Optional[List[str]] = None,
) -> None:
"""Run cc with the given user config(s). Expect errors.
Pass extra_options on the command line of cc.
If error is given, the standard error from cc must match this regex.
"""
cp = self.run_with_config(crypto_user_config, mbedtls_user_config,
extra_options=extra_options)
self.cc_output = cp.stdout
if error is not None:
# Assert the error text before the status. That way, if it fails,
# we see the unexpected error messages in the test log.
self.assertRegex(cp.stderr, error)
self.assertGreater(cp.returncode, 0)
self.assertLess(cp.returncode, 126)
self.cc_output = None
# Nominal case, run first
def test_01_nominal(self) -> None:
self.good_case(None)
# Trivial error case, run second
def test_02_error(self) -> None:
self.bad_case('#error "Bad crypto configuration"',
error='"Bad crypto configuration"')

186
vendor/mbedtls/framework/scripts/output_env.sh vendored Executable file
View File

@@ -0,0 +1,186 @@
#! /usr/bin/env sh
# output_env.sh
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
# Purpose
#
# To print out all the relevant information about the development environment.
#
# This includes:
# - architecture of the system
# - type and version of the operating system
# - version of make and cmake
# - version of armcc, clang, gcc-arm and gcc compilers
# - version of libc, clang, asan and valgrind if installed
# - version of gnuTLS and OpenSSL
print_version()
{
BIN="$1"
shift
ARGS="$1"
shift
VARIANT="$1"
shift
if [ -n "$VARIANT" ]; then
VARIANT=" ($VARIANT)"
fi
if ! type "$BIN" > /dev/null 2>&1; then
echo " * ${BIN##*/}$VARIANT: Not found."
return 0
fi
BIN=`which "$BIN"`
VERSION_STR=`$BIN $ARGS 2>&1`
# Apply all filters
while [ $# -gt 0 ]; do
FILTER="$1"
shift
VERSION_STR=`echo "$VERSION_STR" | $FILTER`
done
if [ -z "$VERSION_STR" ]; then
VERSION_STR="Version could not be determined."
fi
echo " * ${BIN##*/}$VARIANT: ${BIN} : ${VERSION_STR} "
}
echo "** Platform:"
echo
if [ `uname -s` = "Linux" ]; then
echo "Linux variant"
lsb_release -d -c
else
echo "Unknown Unix variant"
fi
echo
print_version "uname" "-a" ""
echo
echo
echo "** Tool Versions:"
echo
print_version "make" "--version" "" "head -n 1"
echo
print_version "cmake" "--version" "" "head -n 1"
echo
if [ "${RUN_ARMCC:-1}" -ne 0 ]; then
: "${ARMC6_CC:=armclang}"
print_version "$ARMC6_CC" "--vsn" "" "head -n 2"
echo
fi
print_version "arm-none-eabi-gcc" "--version" "" "head -n 1"
echo
print_version "gcc" "--version" "" "head -n 1"
echo
if [ -n "${GCC_EARLIEST+set}" ]; then
print_version "${GCC_EARLIEST}" "--version" "" "head -n 1"
else
echo " GCC_EARLIEST : Not configured."
fi
echo
if [ -n "${GCC_LATEST+set}" ]; then
print_version "${GCC_LATEST}" "--version" "" "head -n 1"
else
echo " GCC_LATEST : Not configured."
fi
echo
print_version "clang" "--version" "" "head -n 2"
echo
if [ -n "${CLANG_EARLIEST+set}" ]; then
print_version "${CLANG_EARLIEST}" "--version" "" "head -n 2"
else
echo " CLANG_EARLIEST : Not configured."
fi
echo
if [ -n "${CLANG_LATEST+set}" ]; then
print_version "${CLANG_LATEST}" "--version" "" "head -n 2"
else
echo " CLANG_LATEST : Not configured."
fi
echo
print_version "ldd" "--version" "" "head -n 1"
echo
print_version "valgrind" "--version" ""
echo
print_version "ctags" "--version" "" "head -n 1"
echo
print_version "gdb" "--version" "" "head -n 1"
echo
print_version "perl" "--version" "" "head -n 2" "grep ."
echo
print_version "python" "--version" "" "head -n 1"
echo
print_version "python3" "--version" "" "head -n 1"
echo
# Find the installed version of Pylint. Installed as a distro package this can
# be pylint3 and as a PEP egg, pylint. In test scripts We prefer pylint over
# pylint3
if type pylint >/dev/null 2>/dev/null; then
print_version "pylint" "--version" "" "sed /^.*config/d" "grep pylint"
elif type pylint3 >/dev/null 2>/dev/null; then
print_version "pylint3" "--version" "" "sed /^.*config/d" "grep pylint"
else
echo " * pylint or pylint3: Not found."
fi
echo
: ${OPENSSL:=openssl}
print_version "$OPENSSL" "version" "default"
echo
if [ -n "${OPENSSL_NEXT+set}" ]; then
print_version "$OPENSSL_NEXT" "version" "next"
else
echo " * openssl (next): Not configured."
fi
echo
: ${GNUTLS_CLI:=gnutls-cli}
print_version "$GNUTLS_CLI" "--version" "default" "head -n 1"
echo
: ${GNUTLS_SERV:=gnutls-serv}
print_version "$GNUTLS_SERV" "--version" "default" "head -n 1"
echo
echo " * Installed asan versions:"
if type dpkg-query >/dev/null 2>/dev/null; then
if ! dpkg-query -f '${Status} ${Package}: ${Version}\n' -W 'libasan*' |
awk '$3 == "installed" && $4 !~ /-/ {print $4, $5}' |
grep .
then
echo " No asan versions installed."
fi
else
echo " Unable to determine the asan version without dpkg."
fi
echo

40
vendor/mbedtls/framework/scripts/pkgconfig.sh vendored Executable file
View File

@@ -0,0 +1,40 @@
#!/bin/sh
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
# Purpose
#
# Test pkgconfig files.
#
# For each of the build pkg-config files, .pc files, check that
# they validate and do some basic sanity testing on the output,
# i.e. that the strings are non-empty.
#
# NOTE: This requires the built pc files to be on the pkg-config
# search path, this can be controlled with env variable
# PKG_CONFIG_PATH. See man(1) pkg-config for details.
#
set -e -u
if [ $# -le 0 ]
then
echo " [!] No package names specified" >&2
echo "Usage: $0 <package name 1> <package name 2> ..." >&2
exit 1
fi
for pc in "$@"; do
printf "testing package config file: ${pc} ... "
pkg-config --validate "${pc}"
version="$(pkg-config --modversion "${pc}")"
test -n "$version"
cflags="$(pkg-config --cflags "${pc}")"
test -n "$cflags"
libs="$(pkg-config --libs "${pc}")"
test -n "$libs"
printf "passed\n"
done
exit 0

View File

@@ -0,0 +1,121 @@
# project-detection.sh
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
# Purpose
#
# This script contains functions for shell scripts to
# help detect which project (Mbed TLS, TF-PSA-Crypto)
# or which Mbed TLS branch they are in.
# Project detection.
#
# Both Mbed TLS and TF-PSA-Cryto repos have a "scripts/project_name.txt" file
# which contains the name of the project. They are used in scripts to know in
# which project/folder we're in.
# This function accepts 2 parameters:
# - $1: boolean value which defines the behavior in case
# "scripts/project_name.txt" is not found:
# - 1: exit with error message
# - 0: simply return an error
# - $2: mandatory value which defined the root folder where to look for
# "scripts/project_name.txt".
read_project_name_file () {
EXIT_IF_NOT_FOUND=$1
ROOT_PATH=$2
PROJECT_NAME_FILE="scripts/project_name.txt"
# Check if file exists.
if [ ! -f "$ROOT_PATH/$PROJECT_NAME_FILE" ]; then
if $EXIT_IF_NOT_FOUND ; then
echo "$ROOT_PATH/$PROJECT_NAME_FILE does not exist... Exiting..." >&2
exit 1
fi
# Simply return a failure in case we were asked not to fail in case of
# missing file.
return 1
fi
if read -r PROJECT_NAME < "$ROOT_PATH/$PROJECT_NAME_FILE"; then :; else
return 1
fi
}
# Check if the current folder is the Mbed TLS root one.
#
# Warning: if this is not run from Mbed TLS/TF-PSA-Crypto root folder, the
# script is terminated with a failure.
in_mbedtls_repo () {
read_project_name_file true .
test "$PROJECT_NAME" = "Mbed TLS"
}
# Check if the current folder is the TF-PSA-Crypto root one.
#
# Warning: if this is not run from Mbed TLS/TF-PSA-Crypto root folder, the
# script is terminated with a failure.
in_tf_psa_crypto_repo () {
read_project_name_file true .
test "$PROJECT_NAME" = "TF-PSA-Crypto"
}
# Check if $1 is an Mbed TLS root folder.
#
# Differently from in_mbedtls_repo() above, this can be run on any folder
# without causing the script to exit.
is_mbedtls_root() {
if ! read_project_name_file false $1 ; then
return 1
fi
test "$PROJECT_NAME" = "Mbed TLS"
}
# Check if $1 is a TF-PSA-Crypto root folder.
#
# Differently from in_tf_psa_crypto_repo() above, this can be run on any folder
# without causing the script to exit.
is_tf_psa_crypto_root() {
if ! read_project_name_file false $1 ; then
return 1
fi
test "$PROJECT_NAME" = "TF-PSA-Crypto"
}
#Branch detection
read_build_info () {
SCRIPT_DIR=$(pwd)
BUILD_INFO_FILE="include/mbedtls/build_info.h"
if [ ! -f "$BUILD_INFO_FILE" ]; then
echo "File $BUILD_INFO_FILE not found."
exit 1
fi
MBEDTLS_VERSION_MAJOR=$(grep "^#define MBEDTLS_VERSION_MAJOR" "$BUILD_INFO_FILE" | awk '{print $3}')
MBEDTLS_VERSION_MINOR=$(grep "^#define MBEDTLS_VERSION_MINOR" "$BUILD_INFO_FILE" | awk '{print $3}')
if [ -z "$MBEDTLS_VERSION_MAJOR" ]; then
echo "MBEDTLS_VERSION_MAJOR not found in $BUILD_INFO_FILE."
exit 1
fi
if [ -z "$MBEDTLS_VERSION_MINOR" ]; then
echo "MBEDTLS_VERSION_MINOR not found in $BUILD_INFO_FILE."
exit 1
fi
}
in_3_6_branch () {
read_build_info
test $MBEDTLS_VERSION_MAJOR = "3" && test $MBEDTLS_VERSION_MINOR = "6"
}
in_4_x_branch () {
read_build_info
test $MBEDTLS_VERSION_MAJOR = "4"
}

View File

@@ -0,0 +1,17 @@
"""Add the consuming repository's scripts to the module search path.
Usage:
import project_scripts # pylint: disable=unused-import
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
import os
import sys
sys.path.append(os.path.join(os.path.dirname(__file__),
os.path.pardir, os.path.pardir,
'scripts'))

19
vendor/mbedtls/framework/scripts/quiet/cmake vendored Executable file
View File

@@ -0,0 +1,19 @@
#! /usr/bin/env bash
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
# This swallows the output of the wrapped tool, unless there is an error.
# This helps reduce excess logging in the CI.
# If you are debugging a build / CI issue, you can get complete unsilenced logs
# by un-commenting the following line (or setting VERBOSE_LOGS in your environment):
# export VERBOSE_LOGS=1
# don't silence invocations containing these arguments
NO_SILENCE=" --version "
TOOL="cmake"
. "$(dirname "$0")/quiet.sh"

19
vendor/mbedtls/framework/scripts/quiet/make vendored Executable file
View File

@@ -0,0 +1,19 @@
#! /usr/bin/env bash
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
# This swallows the output of the wrapped tool, unless there is an error.
# This helps reduce excess logging in the CI.
# If you are debugging a build / CI issue, you can get complete unsilenced logs
# by un-commenting the following line (or setting VERBOSE_LOGS in your environment):
# export VERBOSE_LOGS=1
# don't silence invocations containing these arguments
NO_SILENCE=" --version | test "
TOOL="make"
. "$(dirname "$0")/quiet.sh"

View File

@@ -0,0 +1,79 @@
# -*-mode: sh; sh-shell: bash -*-
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
#
# This swallows the output of the wrapped tool, unless there is an error.
# This helps reduce excess logging in the CI.
# If you are debugging a build / CI issue, you can get complete unsilenced logs
# by un-commenting the following line (or setting VERBOSE_LOGS in your environment):
#
# VERBOSE_LOGS=1
#
# This script provides most of the functionality for the adjacent make and cmake
# wrappers.
#
# It requires two variables to be set:
#
# TOOL - the name of the tool that is being wrapped (with no path), e.g. "make"
#
# NO_SILENCE - a regex that describes the commandline arguments for which output will not
# be silenced, e.g. " --version | test ". In this example, "make lib test" will
# not be silent, but "make lib" will be.
# Identify path to original tool. There is an edge-case here where the quiet wrapper is on the path via
# a symlink or relative path, but "type -ap" yields the wrapper with it's normalised path. We use
# the -ef operator to compare paths, to avoid picking the wrapper in this case (to avoid infinitely
# recursing).
while IFS= read -r ORIGINAL_TOOL; do
if ! [[ $ORIGINAL_TOOL -ef "$0" ]]; then break; fi
done < <(type -ap -- "$TOOL")
print_quoted_args() {
# similar to printf '%q' "$@"
# but produce more human-readable results for common/simple cases like "a b"
for a in "$@"; do
# Get bash to quote the string
printf -v q '%q' "$a"
simple_pattern="^([-[:alnum:]_+./:@]+=)?([^']*)$"
if [[ "$a" != "$q" && $a =~ $simple_pattern ]]; then
# a requires some quoting (a != q), but has no single quotes, so we can
# simplify the quoted form - e.g.:
# a b -> 'a b'
# CFLAGS=a b -> CFLAGS='a b'
q="${BASH_REMATCH[1]}'${BASH_REMATCH[2]}'"
fi
printf " %s" "$q"
done
}
if [[ ! " $* " =~ " --version " ]]; then
# Display the command being invoked - if it succeeds, this is all that will
# be displayed. Don't do this for invocations with --version, because
# this output is often parsed by scripts, so we don't want to modify it.
printf %s "${TOOL}" 1>&2
print_quoted_args "$@" 1>&2
echo 1>&2
fi
if [[ " $@ " =~ $NO_SILENCE || -n "${VERBOSE_LOGS}" ]]; then
# Run original command with no output supression
exec "${ORIGINAL_TOOL}" "$@"
else
# Run original command and capture output & exit status
TMPFILE=$(mktemp "quiet-${TOOL}.XXXXXX")
"${ORIGINAL_TOOL}" "$@" > "${TMPFILE}" 2>&1
EXIT_STATUS=$?
if [[ $EXIT_STATUS -ne 0 ]]; then
# On error, display the full output
cat "${TMPFILE}"
fi
# Remove tmpfile
rm "${TMPFILE}"
# Propagate the exit status
exit $EXIT_STATUS
fi

47
vendor/mbedtls/framework/scripts/recursion.pl vendored Executable file
View File

@@ -0,0 +1,47 @@
#!/usr/bin/env perl
# Find functions making recursive calls to themselves.
# (Multiple recursion where a() calls b() which calls a() not covered.)
#
# When the recursion depth might depend on data controlled by the attacker in
# an unbounded way, those functions should use iteration instead.
#
# Typical usage: framework/scripts/recursion.pl library/*.c
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
use warnings;
use strict;
use utf8;
use open qw(:std utf8);
# exclude functions that are ok:
# - mpi_write_hlp: bounded by size of mbedtls_mpi, a compile-time constant
# - x509_crt_verify_child: bounded by MBEDTLS_X509_MAX_INTERMEDIATE_CA
my $known_ok = qr/mpi_write_hlp|x509_crt_verify_child/;
my $cur_name;
my $inside;
my @funcs;
die "Usage: $0 file.c [...]\n" unless @ARGV;
while (<>)
{
if( /^[^\/#{}\s]/ && ! /\[.*]/ ) {
chomp( $cur_name = $_ ) unless $inside;
} elsif( /^{/ && $cur_name ) {
$inside = 1;
$cur_name =~ s/.* ([^ ]*)\(.*/$1/;
} elsif( /^}/ && $inside ) {
undef $inside;
undef $cur_name;
} elsif( $inside && /\b\Q$cur_name\E\([^)]/ ) {
push @funcs, $cur_name unless /$known_ok/;
}
}
print "$_\n" for @funcs;
exit @funcs;

View File

@@ -0,0 +1,89 @@
#!/bin/sh
help () {
cat <<EOF
Usage: $0 [OPTION] [PLATFORM]...
Run all the metatests whose platform matches any of the given PLATFORM.
A PLATFORM can contain shell wildcards.
Expected output: a lot of scary-looking error messages, since each
metatest is expected to report a failure. The final line should be
"Ran N metatests, all good."
If something goes wrong: the final line should be
"Ran N metatests, X unexpected successes". Look for "Unexpected success"
in the logs above.
-l List the available metatests, don't run them.
EOF
}
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
set -e -u
if [ -d programs ]; then
METATEST_PROGRAM=programs/test/metatest
elif [ -d ../programs ]; then
METATEST_PROGRAM=../programs/test/metatest
elif [ -d ../../programs ]; then
METATEST_PROGRAM=../../programs/test/metatest
else
echo >&2 "$0: FATAL: programs/test/metatest not found"
exit 120
fi
LIST_ONLY=
while getopts hl OPTLET; do
case $OPTLET in
h) help; exit;;
l) LIST_ONLY=1;;
\?) help >&2; exit 120;;
esac
done
shift $((OPTIND - 1))
list_matches () {
while read name platform junk; do
for pattern in "$@"; do
case $platform in
$pattern) echo "$name"; break;;
esac
done
done
}
count=0
errors=0
run_metatest () {
ret=0
"$METATEST_PROGRAM" "$1" || ret=$?
if [ $ret -eq 0 ]; then
echo >&2 "$0: Unexpected success: $1"
errors=$((errors + 1))
fi
count=$((count + 1))
}
# Don't pipe the output of metatest so that if it fails, this script exits
# immediately with a failure status.
full_list=$("$METATEST_PROGRAM" list)
matching_list=$(printf '%s\n' "$full_list" | list_matches "$@")
if [ -n "$LIST_ONLY" ]; then
printf '%s\n' $matching_list
exit
fi
for name in $matching_list; do
run_metatest "$name"
done
if [ $errors -eq 0 ]; then
echo "Ran $count metatests, all good."
exit 0
else
echo "Ran $count metatests, $errors unexpected successes."
exit 1
fi

65
vendor/mbedtls/framework/scripts/run_demos.py vendored Executable file
View File

@@ -0,0 +1,65 @@
#!/usr/bin/env python3
"""Run the Mbed TLS demo scripts.
"""
import argparse
import glob
import subprocess
import sys
def run_demo(demo, quiet=False):
"""Run the specified demo script. Return True if it succeeds."""
args = {}
if quiet:
args['stdout'] = subprocess.DEVNULL
args['stderr'] = subprocess.DEVNULL
returncode = subprocess.call([demo], **args)
return returncode == 0
def run_demos(demos, quiet=False):
"""Run the specified demos and print summary information about failures.
Return True if all demos passed and False if a demo fails.
"""
failures = []
for demo in demos:
if not quiet:
print('#### {} ####'.format(demo))
success = run_demo(demo, quiet=quiet)
if not success:
failures.append(demo)
if not quiet:
print('{}: FAIL'.format(demo))
if quiet:
print('{}: {}'.format(demo, 'PASS' if success else 'FAIL'))
else:
print('')
successes = len(demos) - len(failures)
print('{}/{} demos passed'.format(successes, len(demos)))
if failures and not quiet:
print('Failures:', *failures)
return not failures
def run_all_demos(quiet=False):
"""Run all the available demos.
Return True if all demos passed and False if a demo fails.
"""
mbedtls_demos = glob.glob('programs/*/*_demo.sh')
tf_psa_crypto_demos = glob.glob('tf-psa-crypto/programs/*/*_demo.sh')
all_demos = mbedtls_demos + tf_psa_crypto_demos
if not all_demos:
# Keep the message on one line. pylint: disable=line-too-long
raise Exception('No demos found. run_demos needs to operate from the Mbed TLS toplevel directory.')
return run_demos(all_demos, quiet=quiet)
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--quiet', '-q',
action='store_true',
help="suppress the output of demos")
options = parser.parse_args()
success = run_all_demos(quiet=options.quiet)
sys.exit(0 if success else 1)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,85 @@
#!/bin/sh
usage () {
cat <<EOF
Usage: $0 [OPTION]... COMMIT VERSION
Save historical information about config options.
Record information for the git commit COMMIT (tag, sha or any other refspec).
Store the list of config options and internal macros in file names indicating
the version VERSION.
-C DIR Top-level directory where the git commit can be found.
Default: $project_root
-o DIR Directory for the output files. It must exist.
Default: $history_dir
EOF
}
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
set -eu
framework_root=$(cd "$(dirname "$0")"/.. && pwd)
project_root=${framework_root%/*}
history_dir=$framework_root/history
if [ $# -ne 0 ] && [ "$1" = "--help" ]; then
usage
exit 0
fi
while getopts C:o: OPTLET; do
case $OPTLET in
C) project_root=$OPTARG;;
o) history_dir=$OPTARG;;
*) usage >&2; exit 120;;
esac
done
shift $((OPTIND - 1))
if [ $# -ne 2 ]; then
echo >&2 "This script requires exactly two non-option arguments: COMMIT VERSION"
usage >&2
exit 120
fi
commit=$1
version=$2
# Assert that the git commit exists
git -C "$project_root" cat-file -e "${commit}^{commit}"
if git -C "$project_root" merge-base --is-ancestor 0679e3a841c3317dc1f4a0faacc350bc91662b04 "$commit"; then
product=tfpsacrypto
else
product=mbedtls
fi
temp_file=$(mktemp)
trap 'rm -f "$temp_file"' EXIT INT TERM
## collect_macros VARIANT FILENAME...
## Find the macros defined in the given files. Save the result in
## $history_dir/config-VARIANT-*.txt.
collect_macros () {
output_file="$history_dir/config-$1-$product-$version.txt"
shift
: >"$temp_file"
for header in "$@"; do
git -C "$project_root" show "$commit:$header" >>"$temp_file"
done
sed -n 's![/ ]*# *define *\([A-Z_a-z][0-9A-Z_a-z]*\).*!\1!p' <"$temp_file" |
sort -u >"$output_file"
if [ ! -s "$output_file" ]; then
echo >&2 "$0: Failed to find any config option"
exit 1
fi
}
collect_macros options $(git -C "$project_root" ls-tree -r --name-only "$commit" \
include/mbedtls/mbedtls_config.h include/psa/crypto_config.h)
collect_macros adjust $(git -C "$project_root" ls-tree -r --name-only "$commit" |
grep '_adjust.*\.h$')

View File

@@ -0,0 +1,48 @@
{
"bomFormat": "CycloneDX",
"specVersion": "1.6",
"version": 1,
"metadata": {
"authors": [
{
"name": "@VCS_SBOM_AUTHORS@"
}
]
},
"components": [
{
"type": "library",
"bom-ref": "pkg:github/Mbed-TLS/mbedtls@@VCS_TAG@",
"cpe": "cpe:2.3:a:trustedfirmware:mbed_tls_framework:@VCS_TAG@:*:*:*:*:*:*:*",
"name": "mbedtls-framework",
"version": "@VCS_VERSION@",
"description": "Version-independent build and test framework for TF-PSA-Crypto and Mbed TLS",
"authors": [
{
"name": "@VCS_AUTHORS@"
}
],
"supplier": {
"name": "Trusted Firmware"
},
"licenses": [
{
"license": {
"id": "Apache-2.0"
}
},
{
"license": {
"id": "GPL-2.0-or-later"
}
}
],
"externalReferences": [
{
"type": "vcs",
"url": "https://github.com/Mbed-TLS/mbedtls-framework"
}
]
}
]
}

View File

@@ -0,0 +1,225 @@
#!/usr/bin/env python3
"""Search an outcome file for configurations with given settings.
Read an outcome file and report the configurations in which test_suite_config
runs with the required settings (compilation option enabled or disabled).
"""
import argparse
import os
import re
import subprocess
from typing import Dict, FrozenSet, Iterator, List, Set
import tempfile
import unittest
from mbedtls_framework import build_tree
def make_regexp_for_settings(settings: List[str]) -> str:
"""Construct a regexp matching the interesting outcome lines.
Interesting outcome lines are from test_suite_config where the given
setting is passing.
We assume that the elements of settings don't contain regexp special
characters.
"""
return (r';test_suite_config[^;]*;Config: (' +
'|'.join(settings) +
r');PASS;')
def run_grep(regexp: str, outcome_file: str) -> List[str]:
"""Run grep on the outcome file and return the matching lines."""
env = os.environ.copy()
env['LC_ALL'] = 'C' # Speeds up some versions of GNU grep
try:
return subprocess.check_output(['grep', '-E', regexp, outcome_file],
encoding='ascii',
env=env).splitlines()
except subprocess.CalledProcessError as exn:
if exn.returncode == 1:
return [] # No results. We don't consider this an error.
raise
OUTCOME_LINE_RE = re.compile(r'[^;]*;'
r'([^;]*);'
r'test_suite_config\.(?:[^;]*);'
r'Config: ([^;]*);'
r'PASS;')
def extract_configuration_data(outcome_lines: List[str]) -> Dict[str, FrozenSet[str]]:
"""Extract the configuration data from outcome lines.
The result maps a configuration name to the list of passing settings
in that configuration.
"""
config_data = {} #type: Dict[str, Set[str]]
for line in outcome_lines:
m = OUTCOME_LINE_RE.match(line)
# Assuming a well-formed outcome file, make_regexp_for_settings()
# arranges to only return lines that should match OUTCOME_LINE_RE.
# So this assertion can't fail unless there is an unexpected
# divergence between OUTCOME_LINE_RE, make_regexp_for_settings()
# and the format of the given outcome file
assert m is not None
config_name, setting = m.groups()
if config_name not in config_data:
config_data[config_name] = set()
config_data[config_name].add(setting)
return dict((name, frozenset(settings))
for name, settings in config_data.items())
def matching_configurations(config_data: Dict[str, FrozenSet[str]],
required: List[str]) -> Iterator[str]:
"""Search configurations with the given passing settings.
config_data maps a configuration name to the list of passing settings
in that configuration.
Each setting should be an Mbed TLS compile setting (MBEDTLS_xxx or
PSA_xxx), optionally prefixed with "!".
"""
required_set = frozenset(required)
for config, observed in config_data.items():
if required_set.issubset(observed):
yield config
def search_config_outcomes(outcome_file: str, settings: List[str]) -> List[str]:
"""Search the given outcome file for reports of the given settings.
Each setting should be an Mbed TLS compile setting (MBEDTLS_xxx or
PSA_xxx), optionally prefixed with "!".
"""
# The outcome file is large enough (hundreds of MB) that parsing it
# in Python is slow. Use grep to speed this up considerably.
regexp = make_regexp_for_settings(settings)
outcome_lines = run_grep(regexp, outcome_file)
config_data = extract_configuration_data(outcome_lines)
return sorted(matching_configurations(config_data, settings))
class TestSearch(unittest.TestCase):
"""Tests of search functionality."""
OUTCOME_FILE_CONTENT = """\
whatever;foobar;test_suite_config.part;Config: MBEDTLS_FOO;PASS;
whatever;foobar;test_suite_config.part;Config: !MBEDTLS_FOO;SKIP;
whatever;foobar;test_suite_config.part;Config: MBEDTLS_BAR;PASS;
whatever;foobar;test_suite_config.part;Config: !MBEDTLS_BAR;SKIP;
whatever;foobar;test_suite_config.part;Config: MBEDTLS_QUX;SKIP;
whatever;foobar;test_suite_config.part;Config: !MBEDTLS_QUX;PASS;
whatever;fooqux;test_suite_config.part;Config: MBEDTLS_FOO;PASS;
whatever;fooqux;test_suite_config.part;Config: !MBEDTLS_FOO;SKIP;
whatever;fooqux;test_suite_config.part;Config: MBEDTLS_BAR;SKIP;
whatever;fooqux;test_suite_config.part;Config: !MBEDTLS_BAR;PASS;
whatever;fooqux;test_suite_config.part;Config: MBEDTLS_QUX;PASS;
whatever;fooqux;test_suite_config.part;Config: !MBEDTLS_QUX;SKIP;
whatever;fooqux;test_suite_something.else;Config: MBEDTLS_BAR;PASS;
whatever;boring;test_suite_config.part;Config: BORING;PASS;
whatever;parasite;not_test_suite_config.not;Config: MBEDTLS_FOO;PASS;
whatever;parasite;test_suite_config.but;Config: MBEDTLS_QUX with bells on;PASS;
whatever;parasite;test_suite_config.but;Not Config: MBEDTLS_QUX;PASS;
"""
def search(self, settings: List[str], expected: List[str]) -> None:
"""Test the search functionality.
* settings: settings to search.
* expected: expected search results.
"""
with tempfile.NamedTemporaryFile() as tmp:
tmp.write(self.OUTCOME_FILE_CONTENT.encode())
tmp.flush()
actual = search_config_outcomes(tmp.name, settings)
self.assertEqual(actual, expected)
def test_foo(self) -> None:
self.search(['MBEDTLS_FOO'], ['foobar', 'fooqux'])
def test_bar(self) -> None:
self.search(['MBEDTLS_BAR'], ['foobar'])
def test_foo_bar(self) -> None:
self.search(['MBEDTLS_FOO', 'MBEDTLS_BAR'], ['foobar'])
def test_foo_notbar(self) -> None:
self.search(['MBEDTLS_FOO', '!MBEDTLS_BAR'], ['fooqux'])
class TestOutcome(unittest.TestCase):
"""Tests of outcome file format expectations.
This class builds and runs the config tests in the current configuration.
The configuration must have at least one feature enabled and at least
one feature disabled in each category: MBEDTLS_xxx and PSA_WANT_xxx.
It needs a C compiler.
"""
outcome_content = '' # Let mypy know this field can be used in test case methods
@classmethod
def setUpClass(cls) -> None:
"""Generate, build and run the config tests."""
root_dir = build_tree.guess_project_root()
tests_dir = os.path.join(root_dir, 'tests')
suites = ['test_suite_config.mbedtls_boolean',
'test_suite_config.psa_boolean']
_output = subprocess.check_output(['make'] + suites,
cwd=tests_dir,
stderr=subprocess.STDOUT)
with tempfile.NamedTemporaryFile(dir=tests_dir) as outcome_file:
env = os.environ.copy()
env['MBEDTLS_TEST_PLATFORM'] = 'some_platform'
env['MBEDTLS_TEST_CONFIGURATION'] = 'some_configuration'
env['MBEDTLS_TEST_OUTCOME_FILE'] = outcome_file.name
for suite in suites:
_output = subprocess.check_output([os.path.join(os.path.curdir, suite)],
cwd=tests_dir,
env=env,
stderr=subprocess.STDOUT)
cls.outcome_content = outcome_file.read().decode('ascii')
def test_outcome_format(self) -> None:
"""Check that there are outcome lines matching the expected general format."""
def regex(prefix: str, result: str) -> str:
return (r'(?:\A|\n)some_platform;some_configuration;'
r'test_suite_config\.\w+;Config: {}_\w+;{};'
.format(prefix, result))
self.assertRegex(self.outcome_content, regex('MBEDTLS', 'PASS'))
self.assertRegex(self.outcome_content, regex('MBEDTLS', 'SKIP'))
self.assertRegex(self.outcome_content, regex('!MBEDTLS', 'PASS'))
self.assertRegex(self.outcome_content, regex('!MBEDTLS', 'SKIP'))
self.assertRegex(self.outcome_content, regex('PSA_WANT', 'PASS'))
self.assertRegex(self.outcome_content, regex('PSA_WANT', 'SKIP'))
self.assertRegex(self.outcome_content, regex('!PSA_WANT', 'PASS'))
self.assertRegex(self.outcome_content, regex('!PSA_WANT', 'SKIP'))
def test_outcome_lines(self) -> None:
"""Look for some sample outcome lines."""
def regex(setting: str) -> str:
return (r'(?:\A|\n)some_platform;some_configuration;'
r'test_suite_config\.\w+;Config: {};(PASS|SKIP);'
.format(setting))
self.assertRegex(self.outcome_content, regex('MBEDTLS_AES_C'))
self.assertRegex(self.outcome_content, regex('MBEDTLS_AES_ROM_TABLES'))
self.assertRegex(self.outcome_content, regex('MBEDTLS_SSL_CLI_C'))
self.assertRegex(self.outcome_content, regex('MBEDTLS_X509_CRT_PARSE_C'))
self.assertRegex(self.outcome_content, regex('PSA_WANT_ALG_HMAC'))
self.assertRegex(self.outcome_content, regex('PSA_WANT_KEY_TYPE_AES'))
def main() -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--outcome-file', '-f', metavar='FILE',
default='outcomes.csv',
help='Outcome file to read (default: outcomes.csv)')
parser.add_argument('settings', metavar='SETTING', nargs='+',
help='Required setting (e.g. "MBEDTLS_RSA_C" or "!PSA_WANT_ALG_SHA256")')
options = parser.parse_args()
found = search_config_outcomes(options.outcome_file, options.settings)
for name in found:
print(name)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,175 @@
#!/usr/bin/env python3
"""Test helper for the Mbed TLS configuration file tool
Run config.py with various parameters and write the results to files.
This is a harness to help regression testing, not a functional tester.
Sample usage:
test_config_script.py -d old
## Modify config.py and/or mbedtls_config.h ##
test_config_script.py -d new
diff -ru old new
"""
## Copyright The Mbed TLS Contributors
## SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
##
import argparse
import glob
import os
import re
import shutil
import subprocess
OUTPUT_FILE_PREFIX = 'config-'
def output_file_name(directory, stem, extension):
return os.path.join(directory,
'{}{}.{}'.format(OUTPUT_FILE_PREFIX,
stem, extension))
def cleanup_directory(directory):
"""Remove old output files."""
for extension in []:
pattern = output_file_name(directory, '*', extension)
filenames = glob.glob(pattern)
for filename in filenames:
os.remove(filename)
def prepare_directory(directory):
"""Create the output directory if it doesn't exist yet.
If there are old output files, remove them.
"""
if os.path.exists(directory):
cleanup_directory(directory)
else:
os.makedirs(directory)
def guess_presets_from_help(help_text):
"""Figure out what presets the script supports.
help_text should be the output from running the script with --help.
"""
# Try the output format from config.py
hits = re.findall(r'\{([-\w,]+)\}', help_text)
for hit in hits:
words = set(hit.split(','))
if 'get' in words and 'set' in words and 'unset' in words:
words.remove('get')
words.remove('set')
words.remove('unset')
return words
# Try the output format from config.pl
hits = re.findall(r'\n +([-\w]+) +- ', help_text)
if hits:
return hits
raise Exception("Unable to figure out supported presets. Pass the '-p' option.")
def list_presets(options):
"""Return the list of presets to test.
The list is taken from the command line if present, otherwise it is
extracted from running the config script with --help.
"""
if options.presets:
return re.split(r'[ ,]+', options.presets)
else:
help_text = subprocess.run([options.script, '--help'],
check=False, # config.pl --help returns 255
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT).stdout
return guess_presets_from_help(help_text.decode('ascii'))
def run_one(options, args, stem_prefix='', input_file=None):
"""Run the config script with the given arguments.
Take the original content from input_file if specified, defaulting
to options.input_file if input_file is None.
Write the following files, where xxx contains stem_prefix followed by
a filename-friendly encoding of args:
* config-xxx.h: modified file.
* config-xxx.out: standard output.
* config-xxx.err: standard output.
* config-xxx.status: exit code.
Return ("xxx+", "path/to/config-xxx.h") which can be used as
stem_prefix and input_file to call this function again with new args.
"""
if input_file is None:
input_file = options.input_file
stem = stem_prefix + '-'.join(args)
data_filename = output_file_name(options.output_directory, stem, 'h')
stdout_filename = output_file_name(options.output_directory, stem, 'out')
stderr_filename = output_file_name(options.output_directory, stem, 'err')
status_filename = output_file_name(options.output_directory, stem, 'status')
shutil.copy(input_file, data_filename)
# Pass only the file basename, not the full path, to avoid getting the
# directory name in error messages, which would make comparisons
# between output directories more difficult.
cmd = [os.path.abspath(options.script),
'-f', os.path.basename(data_filename)]
with open(stdout_filename, 'wb') as out:
with open(stderr_filename, 'wb') as err:
status = subprocess.call(cmd + args,
cwd=options.output_directory,
stdin=subprocess.DEVNULL,
stdout=out, stderr=err)
with open(status_filename, 'w') as status_file:
status_file.write('{}\n'.format(status))
return stem + "+", data_filename
### A list of symbols to test with.
### This script currently tests what happens when you change a symbol from
### having a value to not having a value or vice versa. This is not
### necessarily useful behavior, and we may not consider it a bug if
### config.py stops handling that case correctly.
TEST_SYMBOLS = [
'CUSTOM_SYMBOL', # does not exist
'PSA_WANT_KEY_TYPE_AES', # set, no value
'MBEDTLS_MPI_MAX_SIZE', # unset, has a value
'MBEDTLS_NO_UDBL_DIVISION', # unset, in "System support"
'MBEDTLS_PLATFORM_ZEROIZE_ALT', # unset, in "Customisation configuration options"
]
def run_all(options):
"""Run all the command lines to test."""
presets = list_presets(options)
for preset in presets:
run_one(options, [preset])
for symbol in TEST_SYMBOLS:
run_one(options, ['get', symbol])
(stem, filename) = run_one(options, ['set', symbol])
run_one(options, ['get', symbol], stem_prefix=stem, input_file=filename)
run_one(options, ['--force', 'set', symbol])
(stem, filename) = run_one(options, ['set', symbol, 'value'])
run_one(options, ['get', symbol], stem_prefix=stem, input_file=filename)
run_one(options, ['--force', 'set', symbol, 'value'])
run_one(options, ['unset', symbol])
def main():
"""Command line entry point."""
parser = argparse.ArgumentParser(description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
parser.add_argument('-d', metavar='DIR',
dest='output_directory', required=True,
help="""Output directory.""")
parser.add_argument('-f', metavar='FILE',
dest='input_file', default='include/mbedtls/mbedtls_config.h',
help="""Config file (default: %(default)s).""")
parser.add_argument('-p', metavar='PRESET,...',
dest='presets',
help="""Presets to test (default: guessed from --help).""")
parser.add_argument('-s', metavar='FILE',
dest='script', default='scripts/config.py',
help="""Configuration script (default: %(default)s).""")
options = parser.parse_args()
prepare_directory(options.output_directory)
run_all(options)
if __name__ == '__main__':
main()

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,24 @@
#!/usr/bin/env python3
"""Run the PSA Crypto API compliance test suite.
Transitional wrapper to facilitate the migration of consuming branches.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
from typing import List
from mbedtls_framework import psa_compliance
PSA_ARCH_TESTS_REF = 'v23.06_API1.5_ADAC_EAC'
# PSA Compliance tests we expect to fail due to known defects in Mbed TLS /
# TF-PSA-Crypto (or the test suite).
# The test numbers correspond to the numbers used by the console output of the test suite.
# Test number 2xx corresponds to the files in the folder
# psa-arch-tests/api-tests/dev_apis/crypto/test_c0xx
EXPECTED_FAILURES = [] # type: List[int]
if __name__ == '__main__':
psa_compliance.main(PSA_ARCH_TESTS_REF, expected_failures=EXPECTED_FAILURES)

View File

@@ -0,0 +1,211 @@
#!/usr/bin/env python3
"""Test the program psa_constant_names.
Gather constant names from header files and test cases. Compile a C program
to print out their numerical values, feed these numerical values to
psa_constant_names, and check that the output is the original name.
Return 0 if all test cases pass, 1 if the output was not always as expected,
or 1 (with a Python backtrace) if there was an operational error.
"""
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
import argparse
from collections import namedtuple
import os
import re
import subprocess
import sys
from typing import Iterable, List, Optional, Tuple
from mbedtls_framework import build_tree
from mbedtls_framework import c_build_helper
from mbedtls_framework.macro_collector import InputsForTest, PSAMacroEnumerator
from mbedtls_framework import typing_util
def gather_inputs(headers: Iterable[str],
test_suites: Iterable[str],
inputs_class=InputsForTest) -> PSAMacroEnumerator:
"""Read the list of inputs to test psa_constant_names with."""
inputs = inputs_class()
for header in headers:
inputs.parse_header(header)
for test_cases in test_suites:
inputs.parse_test_cases(test_cases)
inputs.add_numerical_values()
inputs.gather_arguments()
return inputs
def run_c(type_word: str,
expressions: Iterable[str],
include_path: Optional[str] = None,
keep_c: bool = False) -> List[str]:
"""Generate and run a program to print out numerical values of C expressions."""
if type_word == 'status':
cast_to = 'long'
printf_format = '%ld'
else:
cast_to = 'unsigned long'
printf_format = '0x%08lx'
return c_build_helper.get_c_expression_values(
cast_to, printf_format,
expressions,
caller='test_psa_constant_names.py for {} values'.format(type_word),
file_label=type_word,
header='#include <psa/crypto.h>',
include_path=include_path,
keep_c=keep_c
)
NORMALIZE_STRIP_RE = re.compile(r'\s+')
def normalize(expr: str) -> str:
"""Normalize the C expression so as not to care about trivial differences.
Currently "trivial differences" means whitespace.
"""
return re.sub(NORMALIZE_STRIP_RE, '', expr)
ALG_TRUNCATED_TO_SELF_RE = \
re.compile(r'PSA_ALG_AEAD_WITH_SHORTENED_TAG\('
r'PSA_ALG_(?:CCM|CHACHA20_POLY1305|GCM)'
r', *16\)\Z')
def is_simplifiable(expr: str) -> bool:
"""Determine whether an expression is simplifiable.
Simplifiable expressions can't be output in their input form, since
the output will be the simple form. Therefore they must be excluded
from testing.
"""
if ALG_TRUNCATED_TO_SELF_RE.match(expr):
return True
return False
def collect_values(inputs: InputsForTest,
type_word: str,
include_path: Optional[str] = None,
keep_c: bool = False) -> Tuple[List[str], List[str]]:
"""Generate expressions using known macro names and calculate their values.
Return a list of pairs of (expr, value) where expr is an expression and
value is a string representation of its integer value.
"""
names = inputs.get_names(type_word)
expressions = sorted(expr
for expr in inputs.generate_expressions(names)
if not is_simplifiable(expr))
values = run_c(type_word, expressions,
include_path=include_path, keep_c=keep_c)
return expressions, values
class Tests:
"""An object representing tests and their results."""
Error = namedtuple('Error',
['type', 'expression', 'value', 'output'])
def __init__(self, options) -> None:
self.options = options
self.count = 0
self.errors = [] #type: List[Tests.Error]
def run_one(self, inputs: InputsForTest, type_word: str) -> None:
"""Test psa_constant_names for the specified type.
Run the program on the names for this type.
Use the inputs to figure out what arguments to pass to macros that
take arguments.
"""
expressions, values = collect_values(inputs, type_word,
include_path=self.options.include,
keep_c=self.options.keep_c)
output_bytes = subprocess.check_output([self.options.program,
type_word] + values)
output = output_bytes.decode('ascii')
outputs = output.strip().split('\n')
self.count += len(expressions)
for expr, value, output in zip(expressions, values, outputs):
if self.options.show:
sys.stdout.write('{} {}\t{}\n'.format(type_word, value, output))
if normalize(expr) != normalize(output):
self.errors.append(self.Error(type=type_word,
expression=expr,
value=value,
output=output))
def run_all(self, inputs: InputsForTest) -> None:
"""Run psa_constant_names on all the gathered inputs."""
for type_word in ['status', 'algorithm', 'ecc_curve', 'dh_group',
'key_type', 'key_usage']:
self.run_one(inputs, type_word)
def report(self, out: typing_util.Writable) -> None:
"""Describe each case where the output is not as expected.
Write the errors to ``out``.
Also write a total.
"""
for error in self.errors:
out.write('For {} "{}", got "{}" (value: {})\n'
.format(error.type, error.expression,
error.output, error.value))
out.write('{} test cases'.format(self.count))
if self.errors:
out.write(', {} FAIL\n'.format(len(self.errors)))
else:
out.write(' PASS\n')
HEADERS = ['psa/crypto.h', 'psa/crypto_extra.h', 'psa/crypto_values.h']
if build_tree.is_mbedtls_3_6():
TEST_SUITES = ['tests/suites/test_suite_psa_crypto_metadata.data']
else:
TEST_SUITES = ['tf-psa-crypto/tests/suites/test_suite_psa_crypto_metadata.data']
def main():
parser = argparse.ArgumentParser(description=globals()['__doc__'])
if build_tree.is_mbedtls_3_6():
parser.add_argument('--include', '-I',
action='append', default=['include'],
help='Directory for header files')
else:
parser.add_argument('--include', '-I',
action='append', default=['tf-psa-crypto/include',
'tf-psa-crypto/drivers/builtin/include',
'tf-psa-crypto/drivers/everest/include',
'tf-psa-crypto/drivers/everest/include/' +
'tf-psa-crypto/private',
'tf-psa-crypto/drivers/pqcp/include',
'include'],
help='Directory for header files')
parser.add_argument('--keep-c',
action='store_true', dest='keep_c', default=False,
help='Keep the intermediate C file')
parser.add_argument('--no-keep-c',
action='store_false', dest='keep_c',
help='Don\'t keep the intermediate C file (default)')
if build_tree.is_mbedtls_3_6():
parser.add_argument('--program',
default='programs/psa/psa_constant_names',
help='Program to test')
else:
parser.add_argument('--program',
default='tf-psa-crypto/programs/psa/psa_constant_names',
help='Program to test')
parser.add_argument('--show',
action='store_true',
help='Show tested values on stdout')
parser.add_argument('--no-show',
action='store_false', dest='show',
help='Don\'t show tested values (default)')
options = parser.parse_args()
headers = [os.path.join(options.include[0], h) for h in HEADERS]
inputs = gather_inputs(headers, TEST_SUITES)
tests = Tests(options)
tests.run_all(inputs)
tests.report(sys.stdout)
if tests.errors:
sys.exit(1)
if __name__ == '__main__':
main()

View File

@@ -0,0 +1,180 @@
#!/usr/bin/env python3
# translate_ciphers.py
#
# Copyright The Mbed TLS Contributors
# SPDX-License-Identifier: Apache-2.0 OR GPL-2.0-or-later
"""
Translate standard ciphersuite names to GnuTLS, OpenSSL and Mbed TLS standards.
To test the translation functions run:
python3 -m unittest translate_cipher.py
"""
import re
import argparse
import unittest
class TestTranslateCiphers(unittest.TestCase):
"""
Ensure translate_ciphers.py translates and formats ciphersuite names
correctly
"""
def test_translate_all_cipher_names(self):
"""
Translate standard ciphersuite names to GnuTLS, OpenSSL and
Mbed TLS counterpart. Use only a small subset of ciphers
that exercise each step of the translation functions
"""
ciphers = [
("TLS_ECDHE_ECDSA_WITH_NULL_SHA",
"+ECDHE-ECDSA:+NULL:+SHA1",
"ECDHE-ECDSA-NULL-SHA",
"TLS-ECDHE-ECDSA-WITH-NULL-SHA"),
("TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
"+ECDHE-ECDSA:+AES-128-GCM:+AEAD",
"ECDHE-ECDSA-AES128-GCM-SHA256",
"TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256"),
("TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA",
"+DHE-RSA:+3DES-CBC:+SHA1",
"EDH-RSA-DES-CBC3-SHA",
"TLS-DHE-RSA-WITH-3DES-EDE-CBC-SHA"),
("TLS_RSA_WITH_AES_256_CBC_SHA",
"+RSA:+AES-256-CBC:+SHA1",
"AES256-SHA",
"TLS-RSA-WITH-AES-256-CBC-SHA"),
("TLS_PSK_WITH_3DES_EDE_CBC_SHA",
"+PSK:+3DES-CBC:+SHA1",
"PSK-3DES-EDE-CBC-SHA",
"TLS-PSK-WITH-3DES-EDE-CBC-SHA"),
("TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
None,
"ECDHE-ECDSA-CHACHA20-POLY1305",
"TLS-ECDHE-ECDSA-WITH-CHACHA20-POLY1305-SHA256"),
("TLS_ECDHE_ECDSA_WITH_AES_128_CCM",
"+ECDHE-ECDSA:+AES-128-CCM:+AEAD",
None,
"TLS-ECDHE-ECDSA-WITH-AES-128-CCM"),
("TLS_ECDHE_RSA_WITH_ARIA_256_GCM_SHA384",
None,
"ECDHE-ARIA256-GCM-SHA384",
"TLS-ECDHE-RSA-WITH-ARIA-256-GCM-SHA384"),
]
for s, g_exp, o_exp, m_exp in ciphers:
if g_exp is not None:
g = translate_gnutls(s)
self.assertEqual(g, g_exp)
if o_exp is not None:
o = translate_ossl(s)
self.assertEqual(o, o_exp)
if m_exp is not None:
m = translate_mbedtls(s)
self.assertEqual(m, m_exp)
def translate_gnutls(s_cipher):
"""
Translate s_cipher from standard ciphersuite naming convention
and return the GnuTLS naming convention
"""
# Replace "_" with "-" to handle ciphersuite names based on Mbed TLS
# naming convention
s_cipher = s_cipher.replace("_", "-")
s_cipher = re.sub(r'\ATLS-', '+', s_cipher)
s_cipher = s_cipher.replace("-WITH-", ":+")
s_cipher = s_cipher.replace("-EDE", "")
# SHA in Mbed TLS == SHA1 GnuTLS,
# if the last 3 chars are SHA append 1
if s_cipher[-3:] == "SHA":
s_cipher = s_cipher+"1"
# CCM or CCM-8 should be followed by ":+AEAD"
# Replace "GCM:+SHAxyz" with "GCM:+AEAD"
if "CCM" in s_cipher or "GCM" in s_cipher:
s_cipher = re.sub(r"GCM-SHA\d\d\d", "GCM", s_cipher)
s_cipher = s_cipher+":+AEAD"
# Replace the last "-" with ":+"
else:
index = s_cipher.rindex("-")
s_cipher = s_cipher[:index] + ":+" + s_cipher[index+1:]
return s_cipher
def translate_ossl(s_cipher):
"""
Translate s_cipher from standard ciphersuite naming convention
and return the OpenSSL naming convention
"""
# Replace "_" with "-" to handle ciphersuite names based on Mbed TLS
# naming convention
s_cipher = s_cipher.replace("_", "-")
s_cipher = re.sub(r'^TLS-', '', s_cipher)
s_cipher = s_cipher.replace("-WITH", "")
# Remove the "-" from "ABC-xyz"
s_cipher = s_cipher.replace("AES-", "AES")
s_cipher = s_cipher.replace("CAMELLIA-", "CAMELLIA")
s_cipher = s_cipher.replace("ARIA-", "ARIA")
# Remove "RSA" if it is at the beginning
s_cipher = re.sub(r'^RSA-', r'', s_cipher)
# For all circumstances outside of PSK
if "PSK" not in s_cipher:
s_cipher = s_cipher.replace("-EDE", "")
s_cipher = s_cipher.replace("3DES-CBC", "DES-CBC3")
# Remove "CBC" if it is not prefixed by DES
s_cipher = re.sub(r'(?<!DES-)CBC-', r'', s_cipher)
# ECDHE-RSA-ARIA does not exist in OpenSSL
s_cipher = s_cipher.replace("ECDHE-RSA-ARIA", "ECDHE-ARIA")
# POLY1305 should not be followed by anything
if "POLY1305" in s_cipher:
index = s_cipher.rindex("POLY1305")
s_cipher = s_cipher[:index+8]
# If DES is being used, Replace DHE with EDH
if "DES" in s_cipher and "DHE" in s_cipher and "ECDHE" not in s_cipher:
s_cipher = s_cipher.replace("DHE", "EDH")
return s_cipher
def translate_mbedtls(s_cipher):
"""
Translate s_cipher from standard ciphersuite naming convention
and return Mbed TLS ciphersuite naming convention
"""
# Replace "_" with "-"
s_cipher = s_cipher.replace("_", "-")
return s_cipher
def format_ciphersuite_names(mode, names):
t = {"g": translate_gnutls,
"o": translate_ossl,
"m": translate_mbedtls
}[mode]
return " ".join(c + '=' + t(c) for c in names)
def main(target, names):
print(format_ciphersuite_names(target, names))
if __name__ == "__main__":
PARSER = argparse.ArgumentParser()
PARSER.add_argument('target', metavar='TARGET', choices=['o', 'g', 'm'])
PARSER.add_argument('names', metavar='NAMES', nargs='+')
ARGS = PARSER.parse_args()
main(ARGS.target, ARGS.names)