Add the GitHub and Gitea steps and split the script into library files

Dry run by default: the script reads from both hosts (tokens, owners,
names, license, SSH) and prints a plan; --apply creates the repositories
and the Gitea -> GitHub push mirror after a final yes. Choosing GitHub
applies the AGPL-3.0 license to the Gitea repository. A failed step is
reported with what exists and how to continue; nothing is ever deleted.

create-project.sh is now the entry point; the work lives in src/lib/, one
responsibility per file. .gitignore gets !src/lib (the Python template
ignores any lib/ folder). Tests grow to 599 checks, with a stub curl and
ssh, and guards for the file structure.

Task: MIL-002#1
Task: MIL-002#2
Task: MIL-002#3
Task: MIL-002#4
Task: MIL-002#5
Task: MIL-002#6
Refs #9
Refs #10
Refs #11
Refs #12
Refs #13
Refs #14

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-05 15:29:10 +08:00
co-authored by Claude Sonnet 5.5
parent 903c948d85
commit 736cdb42fc
30 changed files with 2269 additions and 668 deletions
+3
View File
@@ -184,3 +184,6 @@ cython_debug/
*~
tmp/
repofoundry.*/
!src/lib
+4
View File
@@ -1,2 +1,6 @@
# shellcheck configuration for the RepoFoundry scripts
shell=bash
# follow the "source" lines of create-project.sh into src/lib/; lint from the
# entry point so that variables shared between the files are seen as used
external-sources=true
source-path=SCRIPTDIR
+141 -1
View File
@@ -1,2 +1,142 @@
# repo_foundry
# RepoFoundry
RepoFoundry (`src/create-project.sh`) sets up a new project: a Gitea
repository, optionally an empty GitHub repository with a push mirror from
Gitea to GitHub, and (in a later phase) a local project with the
SQA-QC-Framework.
> **Status: work in progress.** The script can validate its configuration,
> check both hosts and create the repositories and the mirror. Creating the
> local project, and the full installation guide, come in a later phase
> (MIL-003). This file so far documents what is needed to run the host steps
> safely.
## Quick start
```bash
cp config.env.example config.env # service addresses, not secret
cp .env.example .env # credentials: keep private
chmod 600 .env # Linux and macOS
src/create-project.sh # dry run: reads from the hosts, creates nothing
src/create-project.sh --apply # creates the repositories and the mirror
```
Without `--apply` the script only reads from GitHub and Gitea (it checks the
tokens, the owners, the name, the license and SSH) and prints a plan. With
`--apply` it prints the plan again and asks a final question before it creates
anything. Nothing is ever deleted by the script.
Choosing GitHub also applies the AGPL-3.0 license to the Gitea repository, so
that repository is not empty. Without GitHub the Gitea repository is created
empty and has no license.
## Token permissions
Both tokens go in `.env` (never in `config.env`, never in a remote URL). The
script sends them only in a request header, through a private temporary file,
and never prints them.
### GitHub token (`GITHUB_PAT`, only when you choose GitHub)
The same token has two jobs: it creates the repository, and it is the
password Gitea uses to push the mirror. It therefore needs to create
repositories for the chosen owner and to push to the new one.
| Need | Token | Source |
| --- | --- | --- |
| Create a private repository | classic token with the `repo` scope | GitHub REST documentation, "Create a repository" |
| Create a public repository only | classic token with `public_repo` is enough | same |
| Push from the Gitea mirror | covered by `repo` | |
| Check that you belong to the organization owner | probably `read:org` | **Not confirmed**: the GitHub documentation names no scope for this call. If the script says you do not belong to an organization that you do belong to, add `read:org`. |
- **Organization owners:** you must be an active member who is allowed to
create repositories in the organization. Organizations that require SSO or
approval of tokens need the token authorised first.
- **Fine-grained tokens:** the GitHub documentation lists no fine-grained
permission for creating a repository, and this has not been tested.
Use a classic token until it has been.
- **`GITHUB_USER`:** names the account the token belongs to. It is only a
default for the owner prompt; the repository may belong to an organization.
If it differs from the account the token belongs to, the script warns and
uses the account the token belongs to.
### Gitea token (`GITEA_TOKEN`)
| Need | Scope | Source |
| --- | --- | --- |
| Read the account the token belongs to | `read:user` | Gitea documentation |
| Create repositories, manage the push mirror | `write:repository` | Gitea documentation |
| Look up an organization and your permissions in it | `read:organization` | Gitea documentation |
| Create a repository in an organization | probably `write:organization` as well | **Not confirmed**: expected from how the Gitea API groups organization calls; the end-to-end test in MIL-003 will confirm it. |
A missing scope shows up as an HTTP 403 with the server's own message. The
script stops before it creates anything when a preflight check is refused.
## Known limitations
- **The mirror password is stored on the Gitea server.** Gitea needs the
GitHub token to push, so it keeps it. How it is protected depends on the
Gitea version and its administrators. Use a token that is only meant for
this, and revoke it if the Gitea server is ever in doubt.
- **`sync_on_commit` may be ignored.** When a push mirror is created through
the API, some Gitea versions ignore `sync_on_commit` (upstream issue
go-gitea/gitea#22990). The script reads the mirror back and warns if the
setting was not applied; the mirror then syncs on its interval
(`MIRROR_INTERVAL`, default 10 minutes). The first sync is requested right
after the mirror is created.
- **The server decides the shortest interval** and whether push mirrors are
allowed at all. A refused mirror stops the run with the server's message;
the repositories created so far are kept.
- **The license commit.** Gitea adds the license file when the repository is
created with `auto_init`. The script sends no README, so the repository
should hold only `LICENSE`; this is checked in the MIL-003 end-to-end test.
- **No rollback.** If a step fails, the script reports what exists and how to
continue. A repeated run offers to reuse a repository it created earlier
(empty, or in Gitea's case holding only the license). Delete what you do not
want in the web interface.
- **Mirror direction is Gitea to GitHub only.** Push to Gitea; GitHub is a
copy.
- **Requirements:** bash 4.4 or later, `git`, `curl` and `mktemp`; `jq` and
`ssh` are optional. Without `jq` the script reads the few JSON fields it
needs with a simple built-in reader.
## Code layout
`src/create-project.sh` is the entry point: the header, strict mode, loading
and `main`. The work is split by responsibility into `src/lib/`, one job per
file. The files are loaded from that directory only, by a fixed path.
| File | Responsibility |
| --- | --- |
| `constants.sh` | constants and the shared state of a run |
| `output.sh` | messages for the user and redaction of secrets |
| `temp.sh` | private temporary files and their cleanup |
| `util.sh` | small string and list helpers |
| `validate.sh` | validators for names, URLs, tokens, ports, intervals |
| `config.sh` | reading and checking `config.env` and `.env` (never sourced) |
| `tools.sh` | checking the required tools |
| `json.sh` | the little JSON the script reads and writes |
| `http.sh` | the one place that runs `curl`; tokens stay off the command line |
| `api.sh` | GitHub and Gitea API calls and reporting a refused call |
| `prompts.sh` | interactive questions with validation |
| `project.sh` | the project details: asking for them and showing them |
| `hosts.sh` | names and links of the repositories on each host |
| `preflight.sh` | read-only checks of both hosts |
| `steps.sh` | the outcome of each step and the final report |
| `plan.sh` | printing what the script is about to do |
| `repositories.sh` | creating the GitHub and Gitea repositories |
| `mirror.sh` | the Gitea to GitHub push mirror |
| `apply.sh` | confirmations and the apply flow; the only code that changes anything |
| `cli.sh` | usage text and option parsing |
Each file names its responsibility and lists the functions it provides in its
first lines. `tests/test-structure.sh` keeps it that way: every file in
`src/lib/` is loaded, no function is defined twice, and a file does nothing
when it is loaded.
## Development
```bash
bash tests/run-tests.sh # shellcheck, shfmt and all tests, no network
```
+8
View File
@@ -21,3 +21,11 @@ GITEA_URL=https://git.tirsystem.com/
# Gitea REST API base URL. If you leave this out it is GITEA_URL + /api/v1.
GITEA_API_URL=https://git.tirsystem.com/api/v1
# Optional. SSH port of the Gitea server, used for the SSH check and later
# for the framework submodule. Default: 10022.
#GITEA_SSH_PORT=10022
# Optional. How often Gitea pushes to GitHub, as a Go duration (10m0s, 8h0m0s).
# The server may enforce a minimum. Default: 10m0s.
#MIRROR_INTERVAL=10m0s
+97 -634
View File
@@ -4,45 +4,68 @@
# Purpose
# RepoFoundry creates a Gitea repository, optionally an empty GitHub
# repository with a Gitea -> GitHub push mirror, and a local project with
# the SQA-QC-Framework. This version (MIL-001) validates the configuration
# and credentials, checks the required tools and asks for the project
# details. It does NOT contact GitHub or Gitea and changes nothing on disk;
# it only prints a summary of what it collected.
# the SQA-QC-Framework. This version (MIL-002) validates the configuration
# and credentials, asks for the project details, checks both hosts with
# read-only requests (tokens, owners, names, licence, SSH) and, with
# --apply, creates the repositories and the mirror. Choosing GitHub also
# applies the AGPL-3.0 license to the Gitea repository. The local project
# is not created yet.
#
# Dry run by default
# Without --apply the script only reads from GitHub and Gitea (GET
# requests) and prints what it would create. With --apply it prints the plan
# and asks for a final yes before it creates anything. Nothing is ever
# deleted: if a step fails, the script reports what exists and how to
# continue, and a repeated run offers to reuse the empty repositories.
#
# Usage
# create-project.sh [--config FILE] [--env FILE]
# create-project.sh [--apply] [--config FILE] [--env FILE]
# create-project.sh --help | --version
#
# Options
# --apply create the repositories and the mirror (after a final yes)
# --config FILE service addresses (default: config.env in the project root)
# --env FILE credentials (default: .env in the project root)
# -h, --help show this help
# --version show the version
#
# Files (parsed, never sourced)
# config.env GITHUB_API_URL, GITHUB_WEB_URL, GITEA_URL, GITEA_API_URL
# config.env GITHUB_API_URL, GITHUB_WEB_URL, GITEA_URL, GITEA_API_URL and
# the optional GITEA_SSH_PORT (default 10022) and
# MIRROR_INTERVAL (default 10m0s)
# .env GITHUB_PAT, GITHUB_USER, GITEA_TOKEN
#
# Environment
# REPOFOUNDRY_NAME project name used in messages (default: RepoFoundry)
# REPOFOUNDRY_SYNC_WAIT seconds to wait before reading the first mirror
# sync result (default: 3)
# TMPDIR where the private temporary directory is created
#
# Requires
# bash 4.4 or later, git, curl, mktemp; jq is optional (used when present).
# Also the base tools sed, grep, head, tr, rm, rmdir and uname, and stat
# (GNU "stat -c" or BSD "stat -f"; only used outside Windows).
# bash 4.4 or later, git, curl, mktemp; jq and ssh are optional (jq is used
# for JSON when present; ssh is used for the Gitea SSH test).
# Also the base tools sed, grep, head, tr, sleep, rm, rmdir and uname, and
# stat (GNU "stat -c" or BSD "stat -f"; only used outside Windows).
#
# Implements
# MIL-001 tasks 1 to 6 (issues #3 to #8), user story US-001.01 and UC-001
# steps 1 to 3; see docs/. Deviation from the request: its second
# GITEA_URL key is named GITEA_API_URL.
# MIL-001 tasks 1 to 6 and MIL-002 tasks 1 to 5 (issues #3 to #13), user
# stories US-001.01 and US-001.02, UC-001 steps 1 to 7; see docs/. Deviation
# from the request: its second GITEA_URL key is named GITEA_API_URL.
#
# Tracing
# set -x is switched off while the script runs, because a trace would print
# every secret the script handles.
#
# Structure
# This file is the entry point. The work is split by responsibility into
# the files in lib/ next to it (one job per file, see the first lines of
# each file): constants, output, temp, util, validate, config, tools, json,
# http, api, prompts, project, hosts, preflight, steps, plan, repositories,
# mirror, apply and cli. The files are loaded from this directory only.
#
# Exit codes
# 0 success, 1 a failed check or bad input, 2 a usage error.
# 0 success (or a dry run, or a "no" at the final question), 1 a failed
# check, bad input or a failed step, 2 a usage error.
set -Eeuo pipefail
if [[ $- == *x* ]]; then
@@ -55,17 +78,9 @@ if ((BASH_VERSINFO[0] < 4 || (BASH_VERSINFO[0] == 4 && BASH_VERSINFO[1] < 4)));
exit 1
fi
readonly PROJECT_NAME="${REPOFOUNDRY_NAME:-RepoFoundry}"
readonly VERSION="0.1.0"
readonly EXIT_FAILURE=1
readonly EXIT_USAGE=2
readonly MAX_VALUE_LENGTH=2048
readonly MAX_DESCRIPTION_LENGTH=350
readonly HTTP_TIMEOUT_SECONDS=30
# shellcheck disable=SC2034 # read through namerefs (parse_env_file)
readonly CONFIG_KEYS=(GITHUB_API_URL GITHUB_WEB_URL GITEA_URL GITEA_API_URL)
readonly CREDENTIAL_KEYS=(GITHUB_PAT GITHUB_USER GITEA_TOKEN)
# Where this script lives; the library files and the project root are found
# from here, never from the current directory.
readonly SCRIPT_FILE="${BASH_SOURCE[0]}"
case "${BASH_SOURCE[0]}" in
*/*) script_path_dir="${BASH_SOURCE[0]%/*}" ;;
*) script_path_dir="." ;;
@@ -78,622 +93,61 @@ unset script_path_dir
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
readonly PROJECT_ROOT
CONFIG_FILE="$PROJECT_ROOT/config.env"
ENV_FILE="$PROJECT_ROOT/.env"
TMP_DIR=""
HAS_JQ=0
HTTP_STATUS=0
HTTP_BODY_FILE=""
HTTP_ERROR=""
REPLY=""
SECRET_VALUES=()
TEMP_FILES=()
declare -A CONFIG=()
declare -A CREDENTIALS=()
declare -A PROJECT=()
# shellcheck source=lib/constants.sh
source "$SCRIPT_DIR/lib/constants.sh"
# shellcheck source=lib/output.sh
source "$SCRIPT_DIR/lib/output.sh"
# shellcheck source=lib/temp.sh
source "$SCRIPT_DIR/lib/temp.sh"
# shellcheck source=lib/util.sh
source "$SCRIPT_DIR/lib/util.sh"
# shellcheck source=lib/validate.sh
source "$SCRIPT_DIR/lib/validate.sh"
# shellcheck source=lib/config.sh
source "$SCRIPT_DIR/lib/config.sh"
# shellcheck source=lib/tools.sh
source "$SCRIPT_DIR/lib/tools.sh"
# shellcheck source=lib/json.sh
source "$SCRIPT_DIR/lib/json.sh"
# shellcheck source=lib/http.sh
source "$SCRIPT_DIR/lib/http.sh"
# shellcheck source=lib/api.sh
source "$SCRIPT_DIR/lib/api.sh"
# shellcheck source=lib/prompts.sh
source "$SCRIPT_DIR/lib/prompts.sh"
# shellcheck source=lib/project.sh
source "$SCRIPT_DIR/lib/project.sh"
# shellcheck source=lib/hosts.sh
source "$SCRIPT_DIR/lib/hosts.sh"
# shellcheck source=lib/preflight.sh
source "$SCRIPT_DIR/lib/preflight.sh"
# shellcheck source=lib/steps.sh
source "$SCRIPT_DIR/lib/steps.sh"
# shellcheck source=lib/plan.sh
source "$SCRIPT_DIR/lib/plan.sh"
# shellcheck source=lib/repositories.sh
source "$SCRIPT_DIR/lib/repositories.sh"
# shellcheck source=lib/mirror.sh
source "$SCRIPT_DIR/lib/mirror.sh"
# shellcheck source=lib/apply.sh
source "$SCRIPT_DIR/lib/apply.sh"
# shellcheck source=lib/cli.sh
source "$SCRIPT_DIR/lib/cli.sh"
# ---------------------------------------------------------------- output
# redact TEXT: print TEXT with every known secret value replaced.
redact() {
local text="$1" secret
for secret in "${SECRET_VALUES[@]}"; do
if [[ -n $secret ]]; then
text="${text//"$secret"/[redacted]}"
# finish runs on every exit: it reports what a run that started creating
# things did or did not do, then removes the temporary files. It keeps the
# exit status of the run.
finish() {
local code=$?
if ((IS_CREATION_STARTED)); then
report_outcome "$code"
fi
done
printf '%s' "$text"
}
say() {
printf '%s\n' "$(redact "$*")"
}
warn() {
printf 'warning: %s\n' "$(redact "$*")" >&2
}
# die [--code N] MESSAGE: print "error: MESSAGE" and exit (default code 1).
die() {
local code=$EXIT_FAILURE
if [[ ${1:-} == --code ]]; then
code="$2"
shift 2
fi
printf 'error: %s\n' "$(redact "$*")" >&2
exit "$code"
}
usage() {
cat <<EOF
Usage: ${0##*/} [--config FILE] [--env FILE]
${0##*/} --help | --version
EOF
}
usage_error() {
printf 'error: %s\n' "$1" >&2
usage >&2
exit "$EXIT_USAGE"
}
# on_error LINE: report an unexpected failure without echoing the command,
# because a command line could contain a value that must stay private.
on_error() {
printf 'error: unexpected failure near line %s of %s\n' "$1" "${0##*/}" >&2
}
# ------------------------------------------------------- temporary files
# Files are removed one by one and the directory with rmdir: a recursive
# delete is never needed and never used.
cleanup() {
local file
for file in "${TEMP_FILES[@]}"; do
rm -f -- "$file"
done
if [[ -n $TMP_DIR && -d $TMP_DIR ]]; then
# rmdir fails only if something unexpected is left inside; leave it
# rather than delete files this script did not create.
rmdir -- "$TMP_DIR" 2>/dev/null || true
fi
}
setup_temp_dir() {
TMP_DIR="$(umask 077 && mktemp -d "${TMPDIR:-/tmp}/repofoundry.XXXXXX")"
}
# make_temp_file: create a private file in TMP_DIR and return it in REPLY.
make_temp_file() {
REPLY="$(umask 077 && mktemp "$TMP_DIR/file.XXXXXX")"
TEMP_FILES+=("$REPLY")
}
# ------------------------------------------------------------ small helpers
trim() {
local text="$1"
text="${text#"${text%%[![:space:]]*}"}"
text="${text%"${text##*[![:space:]]}"}"
printf '%s' "$text"
}
# in_list NEEDLE ITEM...: succeed if NEEDLE equals one of the items.
in_list() {
local needle="$1" item
shift
for item in "$@"; do
if [[ $item == "$needle" ]]; then
return 0
fi
done
return 1
}
has_control_character() {
[[ $1 == *[[:cntrl:]]* ]]
}
# ------------------------------------------------------------- validators
is_valid_repo_name() {
local name="$1"
[[ $name =~ ^[A-Za-z0-9._-]{1,100}$ ]] || return 1
[[ $name != . && $name != .. && $name != *.git ]]
}
is_valid_gitea_owner() {
[[ $1 =~ ^[A-Za-z0-9][A-Za-z0-9._-]{0,38}$ ]]
}
is_valid_github_owner() {
[[ $1 =~ ^[A-Za-z0-9]([A-Za-z0-9-]{0,37}[A-Za-z0-9])?$ ]]
}
is_valid_description() {
((${#1} <= MAX_DESCRIPTION_LENGTH)) && ! has_control_character "$1"
}
is_valid_directory() {
local path="$1"
[[ -n $path && ${#path} -le 4096 && $path != -* ]] &&
! has_control_character "$path"
}
# https URL without user info, query or fragment, so it can never carry a
# credential.
is_valid_base_url() {
local pattern='^https://[A-Za-z0-9.-]+(:[0-9]{1,5})?(/[A-Za-z0-9._~%+/-]*)?$'
[[ $1 =~ $pattern ]]
}
# Like is_valid_base_url but a query string is allowed (for API requests).
is_valid_request_url() {
local pattern='^https://[A-Za-z0-9.-]+(:[0-9]{1,5})?(/[A-Za-z0-9._~%+/-]*)?(\?[A-Za-z0-9._~%+=&,-]*)?$'
[[ $1 =~ $pattern ]]
}
# Access tokens: no quotes, backslashes or whitespace, so a token cannot
# break out of the curl configuration it is written to.
is_valid_token() {
[[ $1 =~ ^[A-Za-z0-9_.~+/=-]{8,255}$ ]]
}
normalize_url() {
local url="$1"
while [[ $url == */ ]]; do
url="${url%/}"
done
printf '%s' "$url"
}
# --------------------------------------------------- config file parsing
# unquote_value RAW: strip matching quotes (or a trailing " # comment" on an
# unquoted value) and return the value in REPLY. Fails on unbalanced quotes.
unquote_value() {
local raw quote
raw="$(trim "$1")"
quote="${raw:0:1}"
if [[ $quote == '"' || $quote == "'" ]]; then
[[ ${#raw} -ge 2 && ${raw: -1} == "$quote" ]] || return 1
raw="${raw:1:${#raw}-2}"
[[ $raw != *"$quote"* ]] || return 1
else
raw="${raw%%[[:space:]]#*}"
raw="$(trim "$raw")"
[[ $raw != *'"'* && $raw != *"'"* ]] || return 1
fi
REPLY="$raw"
}
# parse_env_file FILE ALLOWED_ARRAY TARGET_ARRAY
# Read KEY=VALUE lines without source or eval. Only keys named in
# ALLOWED_ARRAY are accepted; they are stored in the associative array
# TARGET_ARRAY. Messages name the key and the line, never the value.
parse_env_file() {
local file="$1" line key line_number=0
local pattern='^([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*=(.*)$'
[[ -f $file && -r $file ]] || die "cannot read '$file'"
# shellcheck disable=SC2094 # the loop body only uses $file in messages
while IFS= read -r line || [[ -n $line ]]; do
line_number=$((line_number + 1))
if ((line_number == 1)); then
line="${line#$'\xEF\xBB\xBF'}" # byte order mark from some Windows editors
fi
line="$(trim "${line%$'\r'}")"
if [[ -z $line || $line == \#* ]]; then
continue
fi
[[ $line =~ $pattern ]] ||
die "$file line $line_number: expected KEY=VALUE"
key="${BASH_REMATCH[1]}"
parse_env_entry "$file" "$line_number" "$key" "${BASH_REMATCH[2]}" \
"$2" "$3"
done <"$file"
}
# parse_env_entry FILE LINE KEY RAW_VALUE ALLOWED_ARRAY TARGET_ARRAY
parse_env_entry() {
local file="$1" line_number="$2" key="$3" raw="$4"
local -n allowed_keys="$5"
local -n target_map="$6"
local value
if ! in_list "$key" "${allowed_keys[@]}"; then
if in_list "$key" "${CREDENTIAL_KEYS[@]}"; then
die "$file line $line_number: '$key' is a credential; keep it in the .env file only"
fi
die "$file line $line_number: unknown key '$key'"
fi
if [[ -n ${target_map[$key]+set} ]]; then
die "$file line $line_number: '$key' is set twice"
fi
unquote_value "$raw" ||
die "$file line $line_number: unbalanced or misplaced quotes"
value="$REPLY"
if has_control_character "$value"; then
die "$file line $line_number: '$key' contains a control character"
fi
if ((${#value} > MAX_VALUE_LENGTH)); then
die "$file line $line_number: '$key' is too long"
fi
# shellcheck disable=SC2004 # target_map is an associative array: $key is a string
target_map[$key]="$value"
}
# ------------------------------------------------- configuration checks
validate_config() {
local key url
if [[ -z ${CONFIG[GITEA_URL]:-} ]]; then
die "GITEA_URL is missing in $CONFIG_FILE (see config.env.example)"
fi
CONFIG[GITHUB_API_URL]="${CONFIG[GITHUB_API_URL]:-https://api.github.com}"
CONFIG[GITHUB_WEB_URL]="${CONFIG[GITHUB_WEB_URL]:-https://github.com}"
for key in GITHUB_API_URL GITHUB_WEB_URL GITEA_URL; do
url="$(normalize_url "${CONFIG[$key]}")"
is_valid_base_url "$url" ||
die "$key in $CONFIG_FILE must be an https URL without credentials, query or fragment"
CONFIG[$key]="$url"
done
CONFIG[GITEA_API_URL]="$(normalize_url "${CONFIG[GITEA_API_URL]:-${CONFIG[GITEA_URL]}/api/v1}")"
is_valid_base_url "${CONFIG[GITEA_API_URL]}" ||
die "GITEA_API_URL in $CONFIG_FILE must be an https URL without credentials, query or fragment"
}
validate_credentials() {
if [[ -z ${CREDENTIALS[GITEA_TOKEN]:-} ]]; then
die "GITEA_TOKEN is missing in $ENV_FILE (see .env.example)"
fi
# Register secrets first so that no later message can show them.
SECRET_VALUES+=("${CREDENTIALS[GITEA_TOKEN]}")
if [[ -n ${CREDENTIALS[GITHUB_PAT]:-} ]]; then
SECRET_VALUES+=("${CREDENTIALS[GITHUB_PAT]}")
fi
is_valid_token "${CREDENTIALS[GITEA_TOKEN]}" ||
die "GITEA_TOKEN in $ENV_FILE is not a valid token (8 to 255 letters, digits or _ . ~ + / = -)"
if [[ -n ${CREDENTIALS[GITHUB_PAT]:-} ]] &&
! is_valid_token "${CREDENTIALS[GITHUB_PAT]}"; then
die "GITHUB_PAT in $ENV_FILE is not a valid token (8 to 255 letters, digits or _ . ~ + / = -)"
fi
if [[ -n ${CREDENTIALS[GITHUB_USER]:-} ]] &&
! is_valid_github_owner "${CREDENTIALS[GITHUB_USER]}"; then
die "GITHUB_USER in $ENV_FILE is not a valid GitHub account name"
fi
}
# GitHub credentials are only needed when the Maintainer chose GitHub.
require_github_credentials() {
local key
for key in GITHUB_PAT GITHUB_USER; do
if [[ -z ${CREDENTIALS[$key]:-} ]]; then
die "GitHub was chosen but $key is missing in $ENV_FILE (see .env.example)"
fi
done
}
warn_if_env_unsafe() {
local file="$1" dir mode
case "$(uname -s 2>/dev/null || true)" in
MINGW* | MSYS* | CYGWIN*) ;;
*)
mode="$(stat -c '%a' -- "$file" 2>/dev/null ||
stat -f '%Lp' -- "$file" 2>/dev/null || true)"
if [[ -n $mode ]] && (((8#$mode & 8#077) != 0)); then
warn "$file is readable by other users (mode $mode); run: chmod 600 $file"
fi
;;
esac
dir="."
if [[ $file == */* ]]; then
dir="${file%/*}"
fi
if git -C "$dir" rev-parse --is-inside-work-tree >/dev/null 2>&1 &&
! git -C "$dir" check-ignore -q -- "$file"; then
warn "$file is not ignored by git; add it to .gitignore before committing"
fi
}
load_configuration() {
parse_env_file "$CONFIG_FILE" CONFIG_KEYS CONFIG
validate_config
parse_env_file "$ENV_FILE" CREDENTIAL_KEYS CREDENTIALS
validate_credentials
warn_if_env_unsafe "$ENV_FILE"
}
# -------------------------------------------------------------- tool check
check_tools() {
local tool
local missing=()
for tool in git curl mktemp; do
if ! command -v "$tool" >/dev/null 2>&1; then
missing+=("$tool")
fi
done
if ((${#missing[@]} > 0)); then
die "required tool(s) not found: ${missing[*]}. Install them and try again."
fi
if command -v jq >/dev/null 2>&1; then
HAS_JQ=1
else
HAS_JQ=0
warn "jq not found; using the built-in JSON reader (install jq for stricter parsing)"
fi
}
# --------------------------------------------------------------- JSON
# json_escape TEXT: escape TEXT for use inside a JSON string.
json_escape() {
local text="$1"
text="${text//\\/\\\\}"
text="${text//\"/\\\"}"
text="${text//$'\n'/\\n}"
text="${text//$'\r'/\\r}"
text="${text//$'\t'/\\t}"
printf '%s' "$text"
}
# json_get FILE KEY: print the string, number or boolean value of KEY.
# With jq only the top-level key is read. Without jq the first occurrence of
# the key anywhere in the file is used, which is enough for the flat fields
# the GitHub and Gitea APIs return (name, id, html_url, ...).
json_get() {
local file="$1" key="$2"
[[ $key =~ ^[A-Za-z0-9_]+$ ]] || die "internal error: invalid JSON key"
if ((HAS_JQ)); then
# jq on Windows ends lines with CRLF; strip the CR so values stay clean.
jq -r --arg key "$key" \
'if has($key) and .[$key] != null then .[$key] | tostring else empty end' \
"$file" | tr -d '\r'
else
# grep exits 1 when the key is absent; that is not an error here.
{ grep -o "\"$key\"[[:space:]]*:[[:space:]]*\(\"[^\"]*\"\|[0-9][0-9]*\|true\|false\)" "$file" || true; } |
head -n 1 |
sed -e 's/^[^:]*:[[:space:]]*//' -e 's/^"\(.*\)"$/\1/'
fi
}
# --------------------------------------------------------------- HTTP
describe_http_status() {
case "$1" in
401) printf 'authentication failed: the token is missing, expired or invalid' ;;
403) printf 'the token is valid but not allowed to do this (check its scopes)' ;;
404) printf 'not found (check the name, the owner and the token access)' ;;
409 | 422) printf 'rejected (the name may already exist or be invalid)' ;;
429) printf 'rate limited; wait and try again' ;;
5??) printf 'the server reported an error; try again later' ;;
*) printf 'unexpected HTTP status %s' "$1" ;;
esac
}
describe_curl_error() {
case "$1" in
6) printf 'could not resolve the host name' ;;
7) printf 'could not connect' ;;
28) printf 'the request timed out' ;;
35 | 51 | 58 | 60) printf 'the TLS connection failed' ;;
*) printf 'curl failed with exit code %s' "$1" ;;
esac
}
# http_request METHOD URL SCHEME TOKEN [BODY]
# SCHEME is "token" (Gitea) or "bearer" (GitHub). The token goes into a
# private curl config file, never onto the command line where other users
# could see it. Redirects are not followed, so the token is only ever sent
# to the host named in URL. On success HTTP_STATUS and HTTP_BODY_FILE are
# set; on a network failure the function returns 1 with HTTP_ERROR set.
# shellcheck disable=SC2034 # HTTP_* are results read by the callers
http_request() {
local method="$1" url="$2" scheme="$3" token="$4" body="${5:-}"
local header config_file body_file out_file host curl_status=0
local data_args=()
[[ $method =~ ^(GET|POST|PUT|PATCH|DELETE)$ ]] ||
die "internal error: unsupported HTTP method"
is_valid_request_url "$url" ||
die "refusing to call an invalid or non-https URL"
is_valid_token "$token" || die "refusing to send a malformed token"
case "$scheme" in
token) header="Authorization: token $token" ;;
bearer) header="Authorization: Bearer $token" ;;
*) die "internal error: unknown authentication scheme" ;;
esac
make_temp_file
config_file="$REPLY"
make_temp_file
out_file="$REPLY"
{
printf 'url = "%s"\n' "$url"
printf 'request = "%s"\n' "$method"
printf 'header = "%s"\n' "$header"
printf 'header = "Accept: application/json"\n'
printf 'header = "User-Agent: %s/%s"\n' "$PROJECT_NAME" "$VERSION"
} >"$config_file"
if [[ -n $body ]]; then
make_temp_file
body_file="$REPLY"
printf '%s' "$body" >"$body_file"
printf 'header = "Content-Type: application/json"\n' >>"$config_file"
data_args=(--data-binary "@$body_file")
fi
# curl's own error text is dropped: the exit code is mapped to a message
# that never contains the request.
HTTP_STATUS="$(curl --silent --max-time "$HTTP_TIMEOUT_SECONDS" \
--connect-timeout 10 --output "$out_file" --write-out '%{http_code}' \
--config "$config_file" "${data_args[@]}" 2>/dev/null)" || curl_status=$?
if ((curl_status != 0)); then
host="${url#https://}"
host="${host%%/*}"
HTTP_STATUS=0
HTTP_ERROR="could not reach $host: $(describe_curl_error "$curl_status")"
return 1
fi
HTTP_BODY_FILE="$out_file"
HTTP_ERROR=""
}
# ------------------------------------------------------------- prompts
# prompt_value LABEL DEFAULT VALIDATOR HINT: ask until VALIDATOR accepts the
# answer; the accepted answer is returned in REPLY.
prompt_value() {
local label="$1" default="$2" validator="$3" hint="$4" answer
while true; do
if [[ -n $default ]]; then
printf '%s [%s]: ' "$label" "$default" >&2
else
printf '%s: ' "$label" >&2
fi
IFS= read -r answer || die "no input available for '$label'"
answer="$(trim "$answer")"
answer="${answer:-$default}"
if "$validator" "$answer"; then
REPLY="$answer"
return 0
fi
warn "invalid $label: $hint"
done
}
# prompt_choice LABEL DEFAULT CHOICE...: the answer is returned in REPLY.
prompt_choice() {
local label="$1" default="$2" answer
shift 2
while true; do
printf '%s (%s) [%s]: ' "$label" "$(IFS=/ && echo "$*")" "$default" >&2
IFS= read -r answer || die "no input available for '$label'"
answer="$(trim "$answer")"
answer="${answer:-$default}"
answer="${answer,,}"
if in_list "$answer" "$@"; then
REPLY="$answer"
return 0
fi
warn "invalid $label: choose one of $*"
done
}
# prompt_yes_no LABEL DEFAULT: DEFAULT is y or n; REPLY is 1 (yes) or 0 (no).
prompt_yes_no() {
local label="$1" default="$2" answer
while true; do
printf '%s (y/n) [%s]: ' "$label" "$default" >&2
IFS= read -r answer || die "no input available for '$label'"
answer="$(trim "$answer")"
answer="${answer:-$default}"
case "${answer,,}" in
y | yes)
REPLY=1
return 0
;;
n | no)
REPLY=0
return 0
;;
esac
warn "invalid $label: answer y or n"
done
}
collect_project_details() {
prompt_value "Repository name" "" is_valid_repo_name \
"use letters, digits, '.', '_' or '-' (at most 100), not ending in .git"
PROJECT[name]="$REPLY"
prompt_value "Description (optional)" "" is_valid_description \
"at most $MAX_DESCRIPTION_LENGTH characters and no control characters"
PROJECT[description]="$REPLY"
prompt_choice "Visibility" private private public
PROJECT[visibility]="$REPLY"
prompt_value "Gitea owner (user or organization)" "" is_valid_gitea_owner \
"use letters, digits, '.', '_' or '-' (at most 39)"
PROJECT[gitea_owner]="$REPLY"
prompt_yes_no "Also create a GitHub repository (applies the AGPL license)" y
PROJECT[has_github]="$REPLY"
PROJECT[github_owner]=""
if ((PROJECT[has_github])); then
prompt_value "GitHub owner (user or organization)" \
"${CREDENTIALS[GITHUB_USER]:-}" is_valid_github_owner \
"use letters, digits or '-' (at most 39)"
PROJECT[github_owner]="$REPLY"
fi
prompt_value "Local directory" "./${PROJECT[name]}" is_valid_directory \
"must not be empty, start with '-' or contain control characters"
PROJECT[directory]="$REPLY"
prompt_yes_no "Enable the plan gate" n
PROJECT[is_plan_gate_enabled]="$REPLY"
}
# ------------------------------------------------------------- summary
yes_no() {
if (($1)); then
printf 'yes'
else
printf 'no'
fi
}
credential_state() {
if [[ -n ${CREDENTIALS[$1]:-} ]]; then
printf 'set'
else
printf 'not set'
fi
}
print_summary() {
say ""
say "$PROJECT_NAME $VERSION: nothing has been created yet."
say "Collected details:"
say " Repository : ${PROJECT[name]} (${PROJECT[visibility]})"
say " Description : ${PROJECT[description]:-(none)}"
say " Gitea : ${CONFIG[GITEA_URL]}/${PROJECT[gitea_owner]}/${PROJECT[name]}"
if ((PROJECT[has_github])); then
say " GitHub : ${CONFIG[GITHUB_WEB_URL]}/${PROJECT[github_owner]}/${PROJECT[name]} (AGPL license applied)"
else
say " GitHub : not used"
fi
say " Directory : ${PROJECT[directory]}"
say " Plan gate : $(yes_no "${PROJECT[is_plan_gate_enabled]}")"
say "Credentials : GITEA_TOKEN $(credential_state GITEA_TOKEN)," \
"GITHUB_PAT $(credential_state GITHUB_PAT)"
say "Creating the repositories and the project comes in later phases."
}
# ---------------------------------------------------------------- main
parse_args() {
while (($# > 0)); do
case "$1" in
--config)
(($# >= 2)) || usage_error "--config needs a file"
CONFIG_FILE="$2"
shift 2
;;
--env)
(($# >= 2)) || usage_error "--env needs a file"
ENV_FILE="$2"
shift 2
;;
-h | --help)
sed -n '2,/^set -Eeuo/p' "${BASH_SOURCE[0]}" | sed -e '$d' -e 's/^# \{0,1\}//'
exit 0
;;
--version)
say "$PROJECT_NAME $VERSION"
exit 0
;;
*) usage_error "unknown option: $1" ;;
esac
done
cleanup
}
main() {
trap 'on_error "$LINENO"' ERR
trap cleanup EXIT
trap finish EXIT
is_valid_repo_name "$PROJECT_NAME" ||
die "REPOFOUNDRY_NAME is not a valid project name"
parse_args "$@"
@@ -704,7 +158,16 @@ main() {
if ((PROJECT[has_github])); then
require_github_credentials
fi
init_steps
print_summary
run_preflight
print_plan
if ((IS_APPLY)); then
apply_plan
else
say ""
say "Dry run: nothing was created. Run again with --apply to create it."
fi
}
if [[ ${BASH_SOURCE[0]} == "$0" ]]; then
+61
View File
@@ -0,0 +1,61 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# api.sh - Calls to the GitHub and Gitea APIs and the reporting of a refused call.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: host_label, api_call, server_message, fail_request, expect_status
host_label() {
case "$1" in
gitea) printf 'Gitea' ;;
github) printf 'GitHub' ;;
*) die "internal error: unknown host" ;;
esac
}
# api_call HOST METHOD PATH [BODY]: HOST is gitea or github. A network
# failure ends the run; the HTTP status is left in HTTP_STATUS.
api_call() {
local host="$1" method="$2" path="$3" body="${4:-}"
case "$host" in
gitea)
http_request "$method" "${CONFIG[GITEA_API_URL]}$path" token \
"${CREDENTIALS[GITEA_TOKEN]}" "$body" || die "$HTTP_ERROR"
;;
github)
http_request "$method" "${CONFIG[GITHUB_API_URL]}$path" bearer \
"${CREDENTIALS[GITHUB_PAT]}" "$body" || die "$HTTP_ERROR"
;;
*) die "internal error: unknown host" ;;
esac
}
# server_message: the "message" of the last response, cleaned and shortened.
server_message() {
local message
# A response that is not JSON must not stop the error report.
message="$(json_get "$HTTP_BODY_FILE" message 2>/dev/null || true)"
message="${message//[[:cntrl:]]/ }"
printf '%s' "${message:0:160}"
}
fail_request() {
local detail message
detail="$(describe_http_status "$HTTP_STATUS")"
message="$(server_message)"
die "$1: $detail${message:+ (the server says: $message)}"
}
# expect_status CONTEXT CODE...: go on if the last status is one of CODE,
# otherwise stop with CONTEXT and the server's own words.
expect_status() {
local context="$1" code
shift
for code in "$@"; do
if [[ $HTTP_STATUS == "$code" ]]; then
return 0
fi
done
fail_request "$context"
}
+46
View File
@@ -0,0 +1,46 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# apply.sh - The only code that changes anything: confirmations and the apply flow.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: create_all, confirm_reuse, apply_plan
create_all() {
IS_CREATION_STARTED=1
if ((PROJECT[has_github])); then
create_repository github
fi
create_repository gitea
if ((PROJECT[has_github])); then
configure_mirror
fi
}
confirm_reuse() {
local host
for host in github gitea; do
if ! is_reused "$host"; then
continue
fi
prompt_yes_no "The $(host_label "$host") repository $(repo_url "$host") already exists and has no real content. Reuse it" n
if ! ((REPLY)); then
die "stopped: choose another name or remove the existing repository"
fi
done
if ((${STATE[reuse_gitea]:-0})) && ((PROJECT[has_github])) &&
[[ ${STATE[gitea_repo]} == empty ]]; then
warn "the empty Gitea repository is reused as it is: the $AGPL_LICENSE_KEY license is not added to it"
fi
}
# apply_plan: the only place that changes anything on GitHub or Gitea.
apply_plan() {
prompt_yes_no "Create these now" n
if ! ((REPLY)); then
say "Nothing was created."
return 0
fi
confirm_reuse
create_all
}
+50
View File
@@ -0,0 +1,50 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# cli.sh - The command line: usage text and option parsing.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: usage, usage_error, parse_args
usage() {
cat <<EOF
Usage: ${0##*/} [--apply] [--config FILE] [--env FILE]
${0##*/} --help | --version
EOF
}
usage_error() {
printf 'error: %s\n' "$1" >&2
usage >&2
exit "$EXIT_USAGE"
}
parse_args() {
while (($# > 0)); do
case "$1" in
--apply)
IS_APPLY=1
shift
;;
--config)
(($# >= 2)) || usage_error "--config needs a file"
CONFIG_FILE="$2"
shift 2
;;
--env)
(($# >= 2)) || usage_error "--env needs a file"
ENV_FILE="$2"
shift 2
;;
-h | --help)
sed -n '2,/^set -Eeuo/p' "$SCRIPT_FILE" | sed -e '$d' -e 's/^# \{0,1\}//'
exit 0
;;
--version)
say "$PROJECT_NAME $VERSION"
exit 0
;;
*) usage_error "unknown option: $1" ;;
esac
done
}
+164
View File
@@ -0,0 +1,164 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# config.sh - Reading and checking config.env and .env (parsed, never sourced).
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: unquote_value, parse_env_file, parse_env_entry, validate_config, validate_credentials, require_github_credentials, warn_if_env_unsafe, load_configuration
# unquote_value RAW: strip matching quotes (or a trailing " # comment" on an
# unquoted value) and return the value in REPLY. Fails on unbalanced quotes.
unquote_value() {
local raw quote
raw="$(trim "$1")"
quote="${raw:0:1}"
if [[ $quote == '"' || $quote == "'" ]]; then
[[ ${#raw} -ge 2 && ${raw: -1} == "$quote" ]] || return 1
raw="${raw:1:${#raw}-2}"
[[ $raw != *"$quote"* ]] || return 1
else
raw="${raw%%[[:space:]]#*}"
raw="$(trim "$raw")"
[[ $raw != *'"'* && $raw != *"'"* ]] || return 1
fi
REPLY="$raw"
}
# parse_env_file FILE ALLOWED_ARRAY TARGET_ARRAY
# Read KEY=VALUE lines without source or eval. Only keys named in
# ALLOWED_ARRAY are accepted; they are stored in the associative array
# TARGET_ARRAY. Messages name the key and the line, never the value.
parse_env_file() {
local file="$1" line key line_number=0
local pattern='^([A-Za-z_][A-Za-z0-9_]*)[[:space:]]*=(.*)$'
[[ -f $file && -r $file ]] || die "cannot read '$file'"
# shellcheck disable=SC2094 # the loop body only uses $file in messages
while IFS= read -r line || [[ -n $line ]]; do
line_number=$((line_number + 1))
if ((line_number == 1)); then
line="${line#$'\xEF\xBB\xBF'}" # byte order mark from some Windows editors
fi
line="$(trim "${line%$'\r'}")"
if [[ -z $line || $line == \#* ]]; then
continue
fi
[[ $line =~ $pattern ]] ||
die "$file line $line_number: expected KEY=VALUE"
key="${BASH_REMATCH[1]}"
parse_env_entry "$file" "$line_number" "$key" "${BASH_REMATCH[2]}" \
"$2" "$3"
done <"$file"
}
# parse_env_entry FILE LINE KEY RAW_VALUE ALLOWED_ARRAY TARGET_ARRAY
parse_env_entry() {
local file="$1" line_number="$2" key="$3" raw="$4"
local -n allowed_keys="$5"
local -n target_map="$6"
local value
if ! in_list "$key" "${allowed_keys[@]}"; then
if in_list "$key" "${CREDENTIAL_KEYS[@]}"; then
die "$file line $line_number: '$key' is a credential; keep it in the .env file only"
fi
die "$file line $line_number: unknown key '$key'"
fi
if [[ -n ${target_map[$key]+set} ]]; then
die "$file line $line_number: '$key' is set twice"
fi
unquote_value "$raw" ||
die "$file line $line_number: unbalanced or misplaced quotes"
value="$REPLY"
if has_control_character "$value"; then
die "$file line $line_number: '$key' contains a control character"
fi
if ((${#value} > MAX_VALUE_LENGTH)); then
die "$file line $line_number: '$key' is too long"
fi
# shellcheck disable=SC2004 # target_map is an associative array: $key is a string
target_map[$key]="$value"
}
validate_config() {
local key url
if [[ -z ${CONFIG[GITEA_URL]:-} ]]; then
die "GITEA_URL is missing in $CONFIG_FILE (see config.env.example)"
fi
CONFIG[GITHUB_API_URL]="${CONFIG[GITHUB_API_URL]:-https://api.github.com}"
CONFIG[GITHUB_WEB_URL]="${CONFIG[GITHUB_WEB_URL]:-https://github.com}"
for key in GITHUB_API_URL GITHUB_WEB_URL GITEA_URL; do
url="$(normalize_url "${CONFIG[$key]}")"
is_valid_base_url "$url" ||
die "$key in $CONFIG_FILE must be an https URL without credentials, query or fragment"
CONFIG[$key]="$url"
done
CONFIG[GITEA_API_URL]="$(normalize_url "${CONFIG[GITEA_API_URL]:-${CONFIG[GITEA_URL]}/api/v1}")"
is_valid_base_url "${CONFIG[GITEA_API_URL]}" ||
die "GITEA_API_URL in $CONFIG_FILE must be an https URL without credentials, query or fragment"
CONFIG[GITEA_SSH_PORT]="${CONFIG[GITEA_SSH_PORT]:-$DEFAULT_SSH_PORT}"
is_valid_port "${CONFIG[GITEA_SSH_PORT]}" ||
die "GITEA_SSH_PORT in $CONFIG_FILE must be a port number from 1 to 65535"
CONFIG[MIRROR_INTERVAL]="${CONFIG[MIRROR_INTERVAL]:-$DEFAULT_MIRROR_INTERVAL}"
is_valid_interval "${CONFIG[MIRROR_INTERVAL]}" ||
die "MIRROR_INTERVAL in $CONFIG_FILE must look like 10m0s or 8h0m0s"
}
validate_credentials() {
if [[ -z ${CREDENTIALS[GITEA_TOKEN]:-} ]]; then
die "GITEA_TOKEN is missing in $ENV_FILE (see .env.example)"
fi
# Register secrets first so that no later message can show them.
SECRET_VALUES+=("${CREDENTIALS[GITEA_TOKEN]}")
if [[ -n ${CREDENTIALS[GITHUB_PAT]:-} ]]; then
SECRET_VALUES+=("${CREDENTIALS[GITHUB_PAT]}")
fi
is_valid_token "${CREDENTIALS[GITEA_TOKEN]}" ||
die "GITEA_TOKEN in $ENV_FILE is not a valid token (8 to 255 letters, digits or _ . ~ + / = -)"
if [[ -n ${CREDENTIALS[GITHUB_PAT]:-} ]] &&
! is_valid_token "${CREDENTIALS[GITHUB_PAT]}"; then
die "GITHUB_PAT in $ENV_FILE is not a valid token (8 to 255 letters, digits or _ . ~ + / = -)"
fi
if [[ -n ${CREDENTIALS[GITHUB_USER]:-} ]] &&
! is_valid_github_owner "${CREDENTIALS[GITHUB_USER]}"; then
die "GITHUB_USER in $ENV_FILE is not a valid GitHub account name"
fi
}
# GitHub credentials are only needed when the Maintainer chose GitHub.
require_github_credentials() {
local key
for key in GITHUB_PAT GITHUB_USER; do
if [[ -z ${CREDENTIALS[$key]:-} ]]; then
die "GitHub was chosen but $key is missing in $ENV_FILE (see .env.example)"
fi
done
}
warn_if_env_unsafe() {
local file="$1" dir mode
case "$(uname -s 2>/dev/null || true)" in
MINGW* | MSYS* | CYGWIN*) ;;
*)
mode="$(stat -c '%a' -- "$file" 2>/dev/null ||
stat -f '%Lp' -- "$file" 2>/dev/null || true)"
if [[ -n $mode ]] && (((8#$mode & 8#077) != 0)); then
warn "$file is readable by other users (mode $mode); run: chmod 600 $file"
fi
;;
esac
dir="."
if [[ $file == */* ]]; then
dir="${file%/*}"
fi
if git -C "$dir" rev-parse --is-inside-work-tree >/dev/null 2>&1 &&
! git -C "$dir" check-ignore -q -- "$file"; then
warn "$file is not ignored by git; add it to .gitignore before committing"
fi
}
load_configuration() {
parse_env_file "$CONFIG_FILE" CONFIG_KEYS CONFIG
validate_config
parse_env_file "$ENV_FILE" CREDENTIAL_KEYS CREDENTIALS
validate_credentials
warn_if_env_unsafe "$ENV_FILE"
}
+46
View File
@@ -0,0 +1,46 @@
# shellcheck shell=bash
# constants.sh - Constants and the shared state of a run.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# The state variables are read and written by the other library files; this
# is the one place that declares them.
# shellcheck disable=SC2034 # read and written by the other library files
readonly PROJECT_NAME="${REPOFOUNDRY_NAME:-RepoFoundry}"
readonly VERSION="0.2.0"
readonly EXIT_FAILURE=1
readonly EXIT_USAGE=2
readonly MAX_VALUE_LENGTH=2048
readonly MAX_DESCRIPTION_LENGTH=350
readonly HTTP_TIMEOUT_SECONDS=30
readonly DEFAULT_MIRROR_INTERVAL="10m0s"
readonly DEFAULT_SSH_PORT=10022
readonly AGPL_LICENSE_KEY="AGPL-3.0"
readonly DEFAULT_BRANCH="main"
readonly PLAN_STEPS=("GitHub repository" "Gitea repository" "Push mirror")
# shellcheck disable=SC2034 # read through namerefs (parse_env_file)
readonly CONFIG_KEYS=(GITHUB_API_URL GITHUB_WEB_URL GITEA_URL GITEA_API_URL
GITEA_SSH_PORT MIRROR_INTERVAL)
readonly CREDENTIAL_KEYS=(GITHUB_PAT GITHUB_USER GITEA_TOKEN)
CONFIG_FILE="$PROJECT_ROOT/config.env"
ENV_FILE="$PROJECT_ROOT/.env"
TMP_DIR=""
HAS_JQ=0
HTTP_STATUS=0
HTTP_BODY_FILE=""
HTTP_ERROR=""
REPLY=""
IS_APPLY=0
IS_CREATION_STARTED=0
SECRET_VALUES=()
TEMP_FILES=()
declare -A CONFIG=()
declare -A CREDENTIALS=()
declare -A PROJECT=()
# Facts found by the preflight checks (logins, owner kinds, repository state).
declare -A STATE=()
# Outcome of each step in PLAN_STEPS, for the final report.
declare -A STEP_STATUS=()
declare -A STEP_DETAIL=()
+28
View File
@@ -0,0 +1,28 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# hosts.sh - Names and links of the repositories on each host.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: is_reused, repo_owner, repo_url
# is_reused HOST: succeed if the existing repository on HOST will be reused.
is_reused() {
[[ ${STATE[reuse_$1]:-0} == 1 ]]
}
repo_owner() {
if [[ $1 == gitea ]]; then
printf '%s' "${PROJECT[gitea_owner]}"
else
printf '%s' "${PROJECT[github_owner]}"
fi
}
repo_url() {
if [[ $1 == gitea ]]; then
printf '%s/%s/%s' "${CONFIG[GITEA_URL]}" "${PROJECT[gitea_owner]}" "${PROJECT[name]}"
else
printf '%s/%s/%s' "${CONFIG[GITHUB_WEB_URL]}" "${PROJECT[github_owner]}" "${PROJECT[name]}"
fi
}
+87
View File
@@ -0,0 +1,87 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# http.sh - The one safe place that runs curl: tokens stay off the command line.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: describe_http_status, describe_curl_error, http_request
describe_http_status() {
case "$1" in
401) printf 'authentication failed: the token is missing, expired or invalid' ;;
403) printf 'the token is valid but not allowed to do this (check its scopes)' ;;
404) printf 'not found (check the name, the owner and the token access)' ;;
409 | 422) printf 'rejected (the name may already exist or be invalid)' ;;
429) printf 'rate limited; wait and try again' ;;
5??) printf 'the server reported an error; try again later' ;;
*) printf 'unexpected HTTP status %s' "$1" ;;
esac
}
describe_curl_error() {
case "$1" in
6) printf 'could not resolve the host name' ;;
7) printf 'could not connect' ;;
28) printf 'the request timed out' ;;
35 | 51 | 58 | 60) printf 'the TLS connection failed' ;;
*) printf 'curl failed with exit code %s' "$1" ;;
esac
}
# http_request METHOD URL SCHEME TOKEN [BODY]
# SCHEME is "token" (Gitea) or "bearer" (GitHub). The token goes into a
# private curl config file, never onto the command line where other users
# could see it. Redirects are not followed, so the token is only ever sent
# to the host named in URL. On success HTTP_STATUS and HTTP_BODY_FILE are
# set; on a network failure the function returns 1 with HTTP_ERROR set.
# shellcheck disable=SC2034 # HTTP_* are results read by the callers
http_request() {
local method="$1" url="$2" scheme="$3" token="$4" body="${5:-}"
local header config_file body_file out_file host curl_status=0
local data_args=()
[[ $method =~ ^(GET|POST|PUT|PATCH|DELETE)$ ]] ||
die "internal error: unsupported HTTP method"
is_valid_request_url "$url" ||
die "refusing to call an invalid or non-https URL"
is_valid_token "$token" || die "refusing to send a malformed token"
case "$scheme" in
token) header="Authorization: token $token" ;;
bearer) header="Authorization: Bearer $token" ;;
*) die "internal error: unknown authentication scheme" ;;
esac
make_temp_file
config_file="$REPLY"
make_temp_file
out_file="$REPLY"
{
printf 'url = "%s"\n' "$url"
printf 'request = "%s"\n' "$method"
printf 'header = "%s"\n' "$header"
printf 'header = "Accept: application/json"\n'
printf 'header = "User-Agent: %s/%s"\n' "$PROJECT_NAME" "$VERSION"
} >"$config_file"
if [[ -n $body ]]; then
make_temp_file
body_file="$REPLY"
printf '%s' "$body" >"$body_file"
printf 'header = "Content-Type: application/json"\n' >>"$config_file"
data_args=(--data-binary "@$body_file")
fi
# curl's own error text is dropped: the exit code is mapped to a message
# that never contains the request.
HTTP_STATUS="$(curl --silent --max-time "$HTTP_TIMEOUT_SECONDS" \
--connect-timeout 10 --output "$out_file" --write-out '%{http_code}' \
--config "$config_file" "${data_args[@]}" 2>/dev/null)" || curl_status=$?
# The configuration file holds the token and the body may hold another one
# (the mirror password): remove both now instead of at exit.
rm -f -- "$config_file" ${body_file:+"$body_file"}
if ((curl_status != 0)); then
host="${url#https://}"
host="${host%%/*}"
HTTP_STATUS=0
HTTP_ERROR="could not reach $host: $(describe_curl_error "$curl_status")"
return 1
fi
HTTP_BODY_FILE="$out_file"
HTTP_ERROR=""
}
+55
View File
@@ -0,0 +1,55 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# json.sh - Reading and writing the small amount of JSON the script needs.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: json_escape, json_get, json_has_value, json_values
# json_escape TEXT: escape TEXT for use inside a JSON string.
json_escape() {
local text="$1"
text="${text//\\/\\\\}"
text="${text//\"/\\\"}"
text="${text//$'\n'/\\n}"
text="${text//$'\r'/\\r}"
text="${text//$'\t'/\\t}"
printf '%s' "$text"
}
# json_get FILE KEY: print the string, number or boolean value of KEY.
# With jq only the top-level key is read. Without jq the first occurrence of
# the key anywhere in the file is used, which is enough for the flat fields
# the GitHub and Gitea APIs return (name, id, html_url, ...).
json_get() {
local file="$1" key="$2"
[[ $key =~ ^[A-Za-z0-9_]+$ ]] || die "internal error: invalid JSON key"
if ((HAS_JQ)); then
# jq on Windows ends lines with CRLF; strip the CR so values stay clean.
jq -r --arg key "$key" \
'if has($key) and .[$key] != null then .[$key] | tostring else empty end' \
"$file" | tr -d '\r'
else
# grep exits 1 when the key is absent; that is not an error here.
{ grep -o "\"$key\"[[:space:]]*:[[:space:]]*\(\"[^\"]*\"\|[0-9][0-9]*\|true\|false\)" "$file" || true; } |
head -n 1 |
sed -e 's/^[^:]*:[[:space:]]*//' -e 's/^"\(.*\)"$/\1/'
fi
}
# json_has_value FILE KEY VALUE: succeed if the file holds "KEY": "VALUE",
# written with or without a space after the colon.
json_has_value() {
local file="$1" key="$2" value="$3"
grep -Fq "\"$key\":\"$value\"" "$file" ||
grep -Fq "\"$key\": \"$value\"" "$file"
}
# json_values FILE KEY: print every string value of KEY, one per line.
json_values() {
local file="$1" key="$2"
[[ $key =~ ^[A-Za-z0-9_]+$ ]] || die "internal error: invalid JSON key"
# grep exits 1 when the key is absent; that is not an error here.
{ grep -o "\"$key\"[[:space:]]*:[[:space:]]*\"[^\"]*\"" "$file" || true; } |
sed -e 's/^[^:]*:[[:space:]]*//' -e 's/^"\(.*\)"$/\1/'
}
+95
View File
@@ -0,0 +1,95 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# mirror.sh - The Gitea to GitHub push mirror: create, verify, first sync.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: mirror_field, mirror_path, mirror_address, mirror_exists, create_mirror, verify_mirror, request_first_sync, configure_mirror
# mirror_field FILE ADDRESS FIELD: print FIELD of the push mirror whose
# remote_address is ADDRESS. Without jq the first mirror in the list is used,
# which is the only one on a repository this script has just created.
mirror_field() {
local file="$1" address="$2" field="$3"
if ((HAS_JQ)); then
jq -r --arg address "$address" --arg field "$field" \
'[.[] | select(.remote_address == $address)][0]
| if . == null or .[$field] == null then empty else .[$field] | tostring end' \
"$file" | tr -d '\r'
else
json_get "$file" "$field"
fi
}
mirror_path() {
printf '/repos/%s/%s/push_mirrors' "${PROJECT[gitea_owner]}" "${PROJECT[name]}"
}
mirror_address() {
printf '%s/%s/%s.git' "${CONFIG[GITHUB_WEB_URL]}" "${PROJECT[github_owner]}" "${PROJECT[name]}"
}
# mirror_exists ADDRESS: succeed if Gitea already pushes to ADDRESS.
mirror_exists() {
api_call gitea GET "$(mirror_path)"
expect_status "cannot list the push mirrors of the Gitea repository" 200
json_has_value "$HTTP_BODY_FILE" remote_address "$1"
}
create_mirror() {
local address="$1" body
# The GitHub token is the password of the mirror; the body file that holds
# it is removed as soon as the request has been sent.
body="$(printf '{"remote_address":"%s","remote_username":"%s","remote_password":"%s","interval":"%s","sync_on_commit":true}' \
"$address" "${STATE[github_login]}" "${CREDENTIALS[GITHUB_PAT]}" \
"${CONFIG[MIRROR_INTERVAL]}")"
api_call gitea POST "$(mirror_path)" "$body"
expect_status "cannot create the push mirror (push mirrors may be switched off on the Gitea server, the interval may be shorter than the server allows, or the GitHub token may not push to the new repository)" 200 201
}
# verify_mirror ADDRESS: read the mirror back and report what Gitea applied.
verify_mirror() {
local address="$1" on_commit
api_call gitea GET "$(mirror_path)"
expect_status "cannot read the push mirror back from Gitea" 200
json_has_value "$HTTP_BODY_FILE" remote_address "$address" ||
die "Gitea did not list the push mirror after creating it"
on_commit="$(mirror_field "$HTTP_BODY_FILE" "$address" sync_on_commit)"
if [[ $on_commit != true ]]; then
warn "Gitea did not apply sync_on_commit (a known server issue): the mirror syncs every ${CONFIG[MIRROR_INTERVAL]}, not on every commit. Switch it on in the repository settings if you need it."
STEP_DETAIL["Push mirror"]="(syncs every ${CONFIG[MIRROR_INTERVAL]}, not on every commit)"
fi
}
# request_first_sync ADDRESS: ask Gitea for a first push and report an error
# it records. A failure here is a warning: the mirror retries by itself.
request_first_sync() {
local address="$1" last_error
api_call gitea POST "$(mirror_path)-sync"
if [[ $HTTP_STATUS != 200 && $HTTP_STATUS != 204 ]]; then
warn "could not ask Gitea for the first sync (HTTP $HTTP_STATUS); it runs at the next interval"
return 0
fi
sleep "${REPOFOUNDRY_SYNC_WAIT:-3}"
api_call gitea GET "$(mirror_path)"
if [[ $HTTP_STATUS == 200 ]]; then
last_error="$(mirror_field "$HTTP_BODY_FILE" "$address" last_error)"
if [[ -n $last_error ]]; then
warn "the first mirror sync reported: ${last_error:0:200}"
fi
fi
}
configure_mirror() {
local label="Push mirror" address
address="$(mirror_address)"
begin_step "$label"
if mirror_exists "$address"; then
finish_step "$label" "reused" "Gitea -> $address"
else
create_mirror "$address"
finish_step "$label" "created" "Gitea -> $address"
verify_mirror "$address"
fi
request_first_sync "$address"
}
+43
View File
@@ -0,0 +1,43 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# output.sh - Messages for the user: output, warnings, errors, and redaction of secrets.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: redact, say, warn, die, on_error
# redact TEXT: print TEXT with every known secret value replaced.
redact() {
local text="$1" secret
for secret in "${SECRET_VALUES[@]}"; do
if [[ -n $secret ]]; then
text="${text//"$secret"/[redacted]}"
fi
done
printf '%s' "$text"
}
say() {
printf '%s\n' "$(redact "$*")"
}
warn() {
printf 'warning: %s\n' "$(redact "$*")" >&2
}
# die [--code N] MESSAGE: print "error: MESSAGE" and exit (default code 1).
die() {
local code=$EXIT_FAILURE
if [[ ${1:-} == --code ]]; then
code="$2"
shift 2
fi
printf 'error: %s\n' "$(redact "$*")" >&2
exit "$code"
}
# on_error LINE: report an unexpected failure without echoing the command,
# because a command line could contain a value that must stay private.
on_error() {
printf 'error: unexpected failure near line %s of %s\n' "$1" "${0##*/}" >&2
}
+39
View File
@@ -0,0 +1,39 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# plan.sh - Printing what the script is about to do.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: print_plan
print_plan() {
local gitea_action github_action origin_note
say ""
say "Plan:"
if ((STATE[reuse_gitea])); then
gitea_action="reuse the existing repository (you will be asked to confirm)"
elif ((PROJECT[has_github])); then
gitea_action="create (${PROJECT[visibility]}) with the $AGPL_LICENSE_KEY license"
else
gitea_action="create (${PROJECT[visibility]}), empty"
fi
say "$(printf ' %-18s: %s %s' "Gitea repository" "$gitea_action" "$(repo_url gitea)")"
if ((PROJECT[has_github])); then
if ((STATE[reuse_github])); then
github_action="reuse the existing empty repository (you will be asked to confirm)"
else
github_action="create (${PROJECT[visibility]}), empty"
fi
say "$(printf ' %-18s: %s %s' "GitHub repository" "$github_action" "$(repo_url github)")"
say "$(printf ' %-18s: %s' "Push mirror" "Gitea -> GitHub every ${CONFIG[MIRROR_INTERVAL]}")"
else
say "$(printf ' %-18s: %s' "GitHub repository" "not used")"
say "$(printf ' %-18s: %s' "Push mirror" "not used")"
fi
if ((STATE[is_ssh_ok])); then
origin_note="SSH (the SSH test passed)"
else
origin_note="HTTPS (SSH test: ${STATE[ssh_note]})"
fi
say "$(printf ' %-18s: %s' "Local origin" "will use $origin_note, in a later phase")"
}
+167
View File
@@ -0,0 +1,167 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# preflight.sh - Read-only checks of both hosts before anything is created.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: check_gitea_organization, check_gitea_license, inspect_repository, preflight_gitea, check_github_organization, preflight_github, test_gitea_ssh, decide_existing_repositories, run_preflight
check_gitea_organization() {
local org="$1" login="$2"
api_call gitea GET "/orgs/$org"
if [[ $HTTP_STATUS == 404 ]]; then
die "Gitea owner '$org' is neither your account ($login) nor an organization the token can see"
fi
expect_status "cannot look up the Gitea organization '$org'" 200
api_call gitea GET "/users/$login/orgs/$org/permissions"
expect_status "cannot read your permissions in the Gitea organization '$org'" 200
if [[ $(json_get "$HTTP_BODY_FILE" can_create_repository) != true ]]; then
die "you may not create repositories in the Gitea organization '$org'"
fi
}
check_gitea_license() {
api_call gitea GET /licenses
expect_status "cannot list the licenses of the Gitea server" 200
json_has_value "$HTTP_BODY_FILE" key "$AGPL_LICENSE_KEY" ||
die "the Gitea server does not offer the $AGPL_LICENSE_KEY license"
}
# inspect_repository HOST: record in STATE[HOST_repo] whether the repository
# is free (does not exist), empty, license_only or not_empty.
inspect_repository() {
local host="$1" owner name names
owner="$(repo_owner "$host")"
name="${PROJECT[name]}"
api_call "$host" GET "/repos/$owner/$name"
if [[ $HTTP_STATUS == 404 ]]; then
STATE[${host}_repo]="free"
return 0
fi
expect_status "cannot look up the $(host_label "$host") repository $owner/$name" 200
api_call "$host" GET "/repos/$owner/$name/contents"
if [[ $HTTP_STATUS == 404 ]]; then
STATE[${host}_repo]="empty"
return 0
fi
expect_status "cannot read the contents of the $(host_label "$host") repository $owner/$name" 200
names="$(json_values "$HTTP_BODY_FILE" name)"
case "$names" in
"") STATE[${host}_repo]="empty" ;;
LICENSE) STATE[${host}_repo]="license_only" ;;
*) STATE[${host}_repo]="not_empty" ;;
esac
}
preflight_gitea() {
local owner="${PROJECT[gitea_owner]}" login
api_call gitea GET /user
expect_status "Gitea rejected the token" 200
login="$(json_get "$HTTP_BODY_FILE" login)"
[[ -n $login ]] || die "Gitea did not say which account the token belongs to"
STATE[gitea_login]="$login"
if is_same_name "$owner" "$login"; then
STATE[gitea_owner_kind]="user"
else
check_gitea_organization "$owner" "$login"
STATE[gitea_owner_kind]="organization"
fi
if ((PROJECT[has_github])); then
check_gitea_license
fi
inspect_repository gitea
}
check_github_organization() {
local org="$1" login="$2"
api_call github GET "/user/memberships/orgs/$org"
if [[ $HTTP_STATUS == 404 ]]; then
die "GitHub owner '$org' is neither your account ($login) nor an organization you belong to (or the token lacks the read:org scope)"
fi
expect_status "cannot read your membership of the GitHub organization '$org'" 200
if [[ $(json_get "$HTTP_BODY_FILE" state) != active ]]; then
die "your membership of the GitHub organization '$org' is not active"
fi
}
preflight_github() {
local owner="${PROJECT[github_owner]}" configured login
configured="${CREDENTIALS[GITHUB_USER]:-}"
api_call github GET /user
expect_status "GitHub rejected the token" 200
login="$(json_get "$HTTP_BODY_FILE" login)"
[[ -n $login ]] || die "GitHub did not say which account the token belongs to"
STATE[github_login]="$login"
if [[ -n $configured ]] && ! is_same_name "$configured" "$login"; then
warn "GITHUB_USER is '$configured' but the token belongs to '$login'; the mirror will use '$login'"
fi
if is_same_name "$owner" "$login"; then
STATE[github_owner_kind]="user"
else
check_github_organization "$owner" "$login"
STATE[github_owner_kind]="organization"
fi
inspect_repository github
}
# The SSH result decides later whether origin uses SSH or HTTPS. Without ssh
# or without access it is simply "not passed"; it never stops the run.
test_gitea_ssh() {
local host port output status=0
host="${CONFIG[GITEA_URL]#https://}"
host="${host%%/*}"
host="${host%%:*}"
port="${CONFIG[GITEA_SSH_PORT]}"
STATE[is_ssh_ok]=0
STATE[ssh_note]="failed (check your SSH key and that $host:$port is reachable)"
if ! command -v ssh >/dev/null 2>&1; then
STATE[ssh_note]="not tested (ssh is not installed)"
return 0
fi
output="$(ssh -p "$port" -o BatchMode=yes -o ConnectTimeout=5 \
-o StrictHostKeyChecking=yes -T "git@$host" 2>&1)" || status=$?
if ((status == 0)) || [[ $output == *"successfully authenticated"* ]]; then
STATE[is_ssh_ok]=1
STATE[ssh_note]="passed"
fi
}
# decide_existing_repositories: a repository that already exists may only be
# reused when it is empty (Gitea: or holds just the license this script adds).
decide_existing_repositories() {
local host owner state
for host in gitea github; do
STATE[reuse_$host]=0
if [[ $host == github ]] && ! ((PROJECT[has_github])); then
continue
fi
owner="$(repo_owner "$host")"
state="${STATE[${host}_repo]}"
case "$state" in
free) ;;
empty) STATE[reuse_$host]=1 ;;
license_only)
if [[ $host == gitea ]] && ((PROJECT[has_github])); then
STATE[reuse_$host]=1
else
die "the $(host_label "$host") repository $owner/${PROJECT[name]} already exists and has content; choose another name or remove it first"
fi
;;
*)
die "the $(host_label "$host") repository $owner/${PROJECT[name]} already exists and has content; choose another name or remove it first"
;;
esac
done
}
run_preflight() {
say ""
say "Checking the hosts (read-only requests)..."
preflight_gitea
if ((PROJECT[has_github])); then
preflight_github
fi
test_gitea_ssh
decide_existing_repositories
say "All checks passed."
}
+69
View File
@@ -0,0 +1,69 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# project.sh - The details of the project being created: asking for them and showing them.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: collect_project_details, yes_no, credential_state, print_summary
collect_project_details() {
prompt_value "Repository name" "" is_valid_repo_name \
"use letters, digits, '.', '_' or '-' (at most 100), not ending in .git"
PROJECT[name]="$REPLY"
prompt_value "Description (optional)" "" is_valid_description \
"at most $MAX_DESCRIPTION_LENGTH characters and no control characters"
PROJECT[description]="$REPLY"
prompt_choice "Visibility" private private public
PROJECT[visibility]="$REPLY"
prompt_value "Gitea owner (user or organization)" "" is_valid_gitea_owner \
"use letters, digits, '.', '_' or '-' (at most 39)"
PROJECT[gitea_owner]="$REPLY"
prompt_yes_no "Also create a GitHub repository (applies the AGPL license)" y
PROJECT[has_github]="$REPLY"
PROJECT[github_owner]=""
if ((PROJECT[has_github])); then
prompt_value "GitHub owner (user or organization)" \
"${CREDENTIALS[GITHUB_USER]:-}" is_valid_github_owner \
"use letters, digits or '-' (at most 39)"
PROJECT[github_owner]="$REPLY"
fi
prompt_value "Local directory" "./${PROJECT[name]}" is_valid_directory \
"must not be empty, start with '-' or contain control characters"
PROJECT[directory]="$REPLY"
prompt_yes_no "Enable the plan gate" n
PROJECT[is_plan_gate_enabled]="$REPLY"
}
yes_no() {
if (($1)); then
printf 'yes'
else
printf 'no'
fi
}
credential_state() {
if [[ -n ${CREDENTIALS[$1]:-} ]]; then
printf 'set'
else
printf 'not set'
fi
}
print_summary() {
say ""
say "$PROJECT_NAME $VERSION"
say "Collected details:"
say " Repository : ${PROJECT[name]} (${PROJECT[visibility]})"
say " Description : ${PROJECT[description]:-(none)}"
say " Gitea : ${CONFIG[GITEA_URL]}/${PROJECT[gitea_owner]}/${PROJECT[name]}"
if ((PROJECT[has_github])); then
say " GitHub : ${CONFIG[GITHUB_WEB_URL]}/${PROJECT[github_owner]}/${PROJECT[name]} (AGPL license applied)"
else
say " GitHub : not used"
fi
say " Directory : ${PROJECT[directory]}"
say " Plan gate : $(yes_no "${PROJECT[is_plan_gate_enabled]}")"
say "Credentials : GITEA_TOKEN $(credential_state GITEA_TOKEN)," \
"GITHUB_PAT $(credential_state GITHUB_PAT)"
}
+68
View File
@@ -0,0 +1,68 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# prompts.sh - Interactive questions with validation of every answer.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: prompt_value, prompt_choice, prompt_yes_no
# prompt_value LABEL DEFAULT VALIDATOR HINT: ask until VALIDATOR accepts the
# answer; the accepted answer is returned in REPLY.
prompt_value() {
local label="$1" default="$2" validator="$3" hint="$4" answer
while true; do
if [[ -n $default ]]; then
printf '%s [%s]: ' "$label" "$default" >&2
else
printf '%s: ' "$label" >&2
fi
IFS= read -r answer || die "no input available for '$label'"
answer="$(trim "$answer")"
answer="${answer:-$default}"
if "$validator" "$answer"; then
REPLY="$answer"
return 0
fi
warn "invalid $label: $hint"
done
}
# prompt_choice LABEL DEFAULT CHOICE...: the answer is returned in REPLY.
prompt_choice() {
local label="$1" default="$2" answer
shift 2
while true; do
printf '%s (%s) [%s]: ' "$label" "$(IFS=/ && echo "$*")" "$default" >&2
IFS= read -r answer || die "no input available for '$label'"
answer="$(trim "$answer")"
answer="${answer:-$default}"
answer="${answer,,}"
if in_list "$answer" "$@"; then
REPLY="$answer"
return 0
fi
warn "invalid $label: choose one of $*"
done
}
# prompt_yes_no LABEL DEFAULT: DEFAULT is y or n; REPLY is 1 (yes) or 0 (no).
prompt_yes_no() {
local label="$1" default="$2" answer
while true; do
printf '%s (y/n) [%s]: ' "$label" "$default" >&2
IFS= read -r answer || die "no input available for '$label'"
answer="$(trim "$answer")"
answer="${answer:-$default}"
case "${answer,,}" in
y | yes)
REPLY=1
return 0
;;
n | no)
REPLY=0
return 0
;;
esac
warn "invalid $label: answer y or n"
done
}
+47
View File
@@ -0,0 +1,47 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# repositories.sh - Creating the GitHub and Gitea repositories.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: repo_body, create_repository
# repo_body HOST: the JSON body that creates the repository on HOST.
repo_body() {
local host="$1" description private=true extra=""
description="$(json_escape "${PROJECT[description]}")"
if [[ ${PROJECT[visibility]} != private ]]; then
private=false
fi
if [[ $host == github ]]; then
printf '{"name":"%s","description":"%s","private":%s,"auto_init":false}' \
"${PROJECT[name]}" "$description" "$private"
return 0
fi
if ((PROJECT[has_github])); then
extra=',"auto_init":true,"license":"'"$AGPL_LICENSE_KEY"'"'
else
extra=',"auto_init":false'
fi
printf '{"name":"%s","description":"%s","private":%s,"default_branch":"%s"%s}' \
"${PROJECT[name]}" "$description" "$private" "$DEFAULT_BRANCH" "$extra"
}
# create_repository HOST: create the repository, or reuse the existing one.
create_repository() {
local host="$1" label owner path
label="$(host_label "$host") repository"
owner="$(repo_owner "$host")"
if is_reused "$host"; then
finish_step "$label" "reused" "$(repo_url "$host")"
return 0
fi
begin_step "$label"
path="/user/repos"
if [[ ${STATE[${host}_owner_kind]} == organization ]]; then
path="/orgs/$owner/repos"
fi
api_call "$host" POST "$path" "$(repo_body "$host")"
expect_status "cannot create the $label" 201
finish_step "$label" "created" "$(repo_url "$host")"
}
+51
View File
@@ -0,0 +1,51 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# steps.sh - The outcome of each step and the final report of a run.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: init_steps, begin_step, finish_step, report_outcome
init_steps() {
local label
for label in "${PLAN_STEPS[@]}"; do
STEP_STATUS[$label]="not attempted"
STEP_DETAIL[$label]=""
done
if ! ((PROJECT[has_github])); then
STEP_STATUS["GitHub repository"]="not used"
STEP_STATUS["Push mirror"]="not used"
fi
}
# begin_step LABEL marks the step failed until finish_step says otherwise, so
# any stop in the middle of a step is reported as a failure of that step.
begin_step() {
STEP_STATUS[$1]="FAILED"
STEP_DETAIL[$1]=""
}
finish_step() {
STEP_STATUS[$1]="$2"
STEP_DETAIL[$1]="${3:-}"
}
report_outcome() {
local code="$1" label
say ""
if ((code == 0)); then
say "Done. This is what exists now:"
else
say "The run stopped before it finished. This is what exists now:"
fi
for label in "${PLAN_STEPS[@]}"; do
say "$(printf ' %-18s: %s' "$label" "${STEP_STATUS[$label]}${STEP_DETAIL[$label]:+ ${STEP_DETAIL[$label]}}")"
done
if ((code == 0)); then
say "The local project with the framework is created by a later phase."
else
say "To continue: fix the problem named above and run the same command again with --apply."
say "A repository this run created is still empty (or holds only the license), so the next run offers to reuse it."
say "Nothing is deleted automatically. To start over, delete the repositories above in the web interface."
fi
}
+31
View File
@@ -0,0 +1,31 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# temp.sh - Private temporary files and their cleanup (never a recursive delete).
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: cleanup, setup_temp_dir, make_temp_file
# Files are removed one by one and the directory with rmdir: a recursive
# delete is never needed and never used.
cleanup() {
local file
for file in "${TEMP_FILES[@]}"; do
rm -f -- "$file"
done
if [[ -n $TMP_DIR && -d $TMP_DIR ]]; then
# rmdir fails only if something unexpected is left inside; leave it
# rather than delete files this script did not create.
rmdir -- "$TMP_DIR" 2>/dev/null || true
fi
}
setup_temp_dir() {
TMP_DIR="$(umask 077 && mktemp -d "${TMPDIR:-/tmp}/repofoundry.XXXXXX")"
}
# make_temp_file: create a private file in TMP_DIR and return it in REPLY.
make_temp_file() {
REPLY="$(umask 077 && mktemp "$TMP_DIR/file.XXXXXX")"
TEMP_FILES+=("$REPLY")
}
+26
View File
@@ -0,0 +1,26 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# tools.sh - Checking that the required tools are installed.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: check_tools
check_tools() {
local tool
local missing=()
for tool in git curl mktemp; do
if ! command -v "$tool" >/dev/null 2>&1; then
missing+=("$tool")
fi
done
if ((${#missing[@]} > 0)); then
die "required tool(s) not found: ${missing[*]}. Install them and try again."
fi
if command -v jq >/dev/null 2>&1; then
HAS_JQ=1
else
HAS_JQ=0
warn "jq not found; using the built-in JSON reader (install jq for stricter parsing)"
fi
}
+34
View File
@@ -0,0 +1,34 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# util.sh - Small string and list helpers with no knowledge of the project.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: trim, in_list, has_control_character, is_same_name
trim() {
local text="$1"
text="${text#"${text%%[![:space:]]*}"}"
text="${text%"${text##*[![:space:]]}"}"
printf '%s' "$text"
}
# in_list NEEDLE ITEM...: succeed if NEEDLE equals one of the items.
in_list() {
local needle="$1" item
shift
for item in "$@"; do
if [[ $item == "$needle" ]]; then
return 0
fi
done
return 1
}
has_control_character() {
[[ $1 == *[[:cntrl:]]* ]]
}
is_same_name() {
[[ ${1,,} == "${2,,}" ]]
}
+67
View File
@@ -0,0 +1,67 @@
# shellcheck shell=bash
# shellcheck disable=SC2004,SC2034,SC2154 # shared state and arrays are declared in constants.sh
# validate.sh - Validators for names, URLs, tokens, ports and intervals.
#
# Part of create-project.sh: sourced by it, never run on its own.
#
# Provides: is_valid_repo_name, is_valid_gitea_owner, is_valid_github_owner, is_valid_description, is_valid_directory, is_valid_base_url, is_valid_request_url, is_valid_token, is_valid_port, is_valid_interval, normalize_url
is_valid_repo_name() {
local name="$1"
[[ $name =~ ^[A-Za-z0-9._-]{1,100}$ ]] || return 1
[[ $name != . && $name != .. && $name != *.git ]]
}
is_valid_gitea_owner() {
[[ $1 =~ ^[A-Za-z0-9][A-Za-z0-9._-]{0,38}$ ]]
}
is_valid_github_owner() {
[[ $1 =~ ^[A-Za-z0-9]([A-Za-z0-9-]{0,37}[A-Za-z0-9])?$ ]]
}
is_valid_description() {
((${#1} <= MAX_DESCRIPTION_LENGTH)) && ! has_control_character "$1"
}
is_valid_directory() {
local path="$1"
[[ -n $path && ${#path} -le 4096 && $path != -* ]] &&
! has_control_character "$path"
}
# https URL without user info, query or fragment, so it can never carry a
# credential.
is_valid_base_url() {
local pattern='^https://[A-Za-z0-9.-]+(:[0-9]{1,5})?(/[A-Za-z0-9._~%+/-]*)?$'
[[ $1 =~ $pattern ]]
}
# Like is_valid_base_url but a query string is allowed (for API requests).
is_valid_request_url() {
local pattern='^https://[A-Za-z0-9.-]+(:[0-9]{1,5})?(/[A-Za-z0-9._~%+/-]*)?(\?[A-Za-z0-9._~%+=&,-]*)?$'
[[ $1 =~ $pattern ]]
}
# Access tokens: no quotes, backslashes or whitespace, so a token cannot
# break out of the curl configuration it is written to.
is_valid_token() {
[[ $1 =~ ^[A-Za-z0-9_.~+/=-]{8,255}$ ]]
}
is_valid_port() {
[[ $1 =~ ^[0-9]{1,5}$ ]] && ((10#$1 >= 1 && 10#$1 <= 65535))
}
# A Go duration such as 10m0s or 8h0m0s, the form Gitea expects.
is_valid_interval() {
[[ -n $1 && $1 =~ ^([0-9]+h)?([0-9]+m)?([0-9]+s)?$ ]]
}
normalize_url() {
local url="$1"
while [[ $url == */ ]]; do
url="${url%/}"
done
printf '%s' "$url"
}
+134 -10
View File
@@ -11,12 +11,18 @@
# shellcheck disable=SC2016,SC2034 # stub and snippet text is literal on purpose; OUT, ERR and STATUS are read by the test files
REPO_ROOT="$(cd "${BASH_SOURCE[0]%/*}/.." && pwd)"
readonly REPO_ROOT
readonly SCRIPT="$REPO_ROOT/src/create-project.sh"
readonly SRC_DIR="$REPO_ROOT/src"
readonly SCRIPT="$SRC_DIR/create-project.sh"
# Distinctive fake credentials; the tests search all output for them.
readonly FAKE_GITEA_TOKEN="giteaFAKEtoken1234567890"
readonly FAKE_GITHUB_PAT="ghpFAKEtoken1234567890"
# Answers to the prompts: name, description, visibility, Gitea owner, GitHub
# yes or no, GitHub owner, directory, plan gate.
readonly ANSWERS_GITHUB=$'my-app\nA test app\n\nTirSystem\ny\nacme-org\n\nn\n'
readonly ANSWERS_GITEA_ONLY=$'my-app\n\n\nTirSystem\nn\n\nn\n'
TESTS_RUN=0
TESTS_FAILED=0
CURRENT_TEST=""
@@ -59,6 +65,14 @@ assert_not_contains() {
fi
}
# assert_before NAME A B: the line numbers A and B are set and A comes first.
assert_before() {
check
if [[ -z $2 || -z $3 ]] || ((10#$2 >= 10#$3)); then
fail "$1: expected the step at line '$2' before the step at line '$3'"
fi
}
assert_file_exists() {
check
if [[ ! -e $2 ]]; then
@@ -111,25 +125,133 @@ write_stub() {
chmod +x "$WORK/bin/$1"
}
# write_curl_stub: a curl that records its arguments and configuration and
# answers with STUB_CURL_STATUS (default 200) and body STUB_CURL_BODY.
# write_curl_stub: a curl that records every call and answers from the
# routes file in the work directory (see write_routes). Without a routes file
# it answers STUB_CURL_STATUS (default 200) with the body STUB_CURL_BODY.
# Records: curl.args (all arguments), curl.config (the private config file),
# curl.calls ("METHOD URL" per call) and curl.bodies (each request body).
write_curl_stub() {
write_stub curl '
printf "%s\n" "$@" >>"$STUB_DIR/curl.args"
out="" cfg=""
cat >"$WORK/bin/curl" <<'STUB'
#!/usr/bin/env bash
printf '%s\n' "$@" >>"$STUB_DIR/curl.args"
out="" cfg="" data=""
while (($# > 0)); do
case "$1" in
--output) out="$2"; shift 2 ;;
--config) cfg="$2"; shift 2 ;;
--data-binary) data="${2#@}"; shift 2 ;;
*) shift ;;
esac
done
if [[ -n $cfg ]]; then cat "$cfg" >>"$STUB_DIR/curl.config"; fi
url="" method=""
if [[ -n $cfg ]]; then
cat "$cfg" >>"$STUB_DIR/curl.config"
url="$(sed -n 's/^url = "\(.*\)"$/\1/p' "$cfg")"
method="$(sed -n 's/^request = "\(.*\)"$/\1/p' "$cfg")"
fi
printf '%s %s\n' "$method" "$url" >>"$STUB_DIR/curl.calls"
if [[ -n $data && -f $data ]]; then
printf '%s %s\n%s\n' "$method" "$url" "$(cat "$data")" >>"$STUB_DIR/curl.bodies"
fi
status="${STUB_CURL_STATUS:-200}"
body="${STUB_CURL_BODY:-}"
if [[ -z $body ]]; then body="{\"ok\":true}"; fi
if [[ -n $out ]]; then printf "%s" "$body" >"$out"; fi
printf "%s" "${STUB_CURL_STATUS:-200}"
exit "${STUB_CURL_EXIT:-0}"'
if [[ -f $STUB_DIR/routes ]]; then
status=404
body="{\"message\":\"Not Found\"}"
n=0 first_unused="" last_match=""
while IFS='|' read -r m pattern st rest; do
n=$((n + 1))
if [[ $m == "$method" && $url == *"$pattern" ]]; then
last_match=$n
if [[ -z $first_unused ]] && ! grep -qx "$n" "$STUB_DIR/routes.used" 2>/dev/null; then
first_unused=$n
fi
fi
done <"$STUB_DIR/routes"
pick="${first_unused:-$last_match}"
if [[ -n $pick ]]; then
if [[ -n $first_unused ]]; then printf '%s\n' "$pick" >>"$STUB_DIR/routes.used"; fi
IFS='|' read -r _ _ status body <<<"$(sed -n "${pick}p" "$STUB_DIR/routes")"
fi
fi
if [[ $status == exit* ]]; then exit "${status#exit}"; fi
if [[ -n $out ]]; then printf '%s' "$body" >"$out"; fi
printf '%s' "$status"
exit "${STUB_CURL_EXIT:-0}"
STUB
chmod +x "$WORK/bin/curl"
}
# write_ssh_stub [EXIT]: an ssh that records its arguments and, by default,
# answers like a Gitea server that accepted the key.
write_ssh_stub() {
cat >"$WORK/bin/ssh" <<STUB
#!/usr/bin/env bash
printf '%s\n' "\$@" >>"\$STUB_DIR/ssh.args"
if [[ ${1:-0} == 0 ]]; then
echo "Hi there, gitea-user! You've successfully authenticated, but Gitea does not provide shell access."
else
echo "Permission denied (publickey)."
fi
exit ${1:-0}
STUB
chmod +x "$WORK/bin/ssh"
}
# write_routes: read the routes for the stub curl from stdin. One route per
# line: METHOD|URL-SUFFIX|STATUS|BODY. A request takes the first matching
# route that was not used yet, so repeated calls can get different answers;
# when all are used, the last match answers again. STATUS "exitN" makes curl
# fail with exit code N.
write_routes() {
cat >"$WORK/routes"
: >"$WORK/routes.used"
}
# prepend_route LINE: answer a request before the routes already written.
prepend_route() {
local rest
rest="$(cat "$WORK/routes")"
printf '%s\n%s\n' "$1" "$rest" >"$WORK/routes"
# The lines moved, so the record of used routes no longer applies.
: >"$WORK/routes.used"
}
# write_happy_routes: both hosts accept the tokens; Gitea owner TirSystem and
# GitHub owner acme-org are organizations; nothing exists yet.
write_happy_routes() {
write_routes <<'ROUTES'
GET|/api/v1/user|200|{"login":"gitea-user"}
GET|/api/v1/orgs/TirSystem|200|{"username":"TirSystem"}
GET|/api/v1/users/gitea-user/orgs/TirSystem/permissions|200|{"can_create_repository":true,"is_owner":true}
GET|/api/v1/licenses|200|[{"key":"AGPL-3.0","name":"AGPL-3.0"}]
GET|/api/v1/repos/TirSystem/my-app|404|{"message":"not found"}
GET|api.github.com/user|200|{"login":"octo-user"}
GET|api.github.com/user/memberships/orgs/acme-org|200|{"state":"active","role":"member"}
GET|api.github.com/repos/acme-org/my-app|404|{"message":"Not Found"}
GET|/api/v1/repos/TirSystem/my-app/push_mirrors|200|[]
GET|/api/v1/repos/TirSystem/my-app/push_mirrors|200|[{"remote_address":"https://github.com/acme-org/my-app.git","sync_on_commit":true,"interval":"10m0s","last_error":""}]
POST|api.github.com/orgs/acme-org/repos|201|{"html_url":"https://github.com/acme-org/my-app"}
POST|/api/v1/orgs/TirSystem/repos|201|{"html_url":"https://git.example.test/TirSystem/my-app"}
POST|/api/v1/repos/TirSystem/my-app/push_mirrors|200|{"remote_address":"https://github.com/acme-org/my-app.git"}
POST|/api/v1/repos/TirSystem/my-app/push_mirrors-sync|200|{}
ROUTES
}
# setup_hosts: fixtures, stub curl and ssh, and the happy routes.
setup_hosts() {
write_fixtures
write_curl_stub
write_ssh_stub 0
write_happy_routes
}
# calls: the "METHOD URL" lines the stub curl received (empty if none).
calls() {
if [[ -f $WORK/curl.calls ]]; then
cat "$WORK/curl.calls"
fi
}
# run_cli STDIN ARGS...: run create-project.sh with answers from STDIN (a
@@ -139,6 +261,7 @@ run_cli() {
shift
STATUS=0
PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" TMPDIR="$WORK/tmp" \
REPOFOUNDRY_SYNC_WAIT=0 \
"$BASH" "$SCRIPT" "$@" <<<"$input" >"$WORK/out.txt" 2>"$WORK/err.txt" ||
STATUS=$?
OUT="$(cat "$WORK/out.txt")"
@@ -155,6 +278,7 @@ run_lib() {
printf '%s\n' "$2"
} >"$WORK/snippet.sh"
PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" TMPDIR="$WORK/tmp" \
REPOFOUNDRY_SYNC_WAIT=0 \
"$BASH" "$WORK/snippet.sh" <<<"$input" >"$WORK/out.txt" 2>"$WORK/err.txt" ||
STATUS=$?
OUT="$(cat "$WORK/out.txt")"
+8 -3
View File
@@ -23,15 +23,20 @@ failed_checks=0
run_static_checks() {
printf '== static checks\n'
bash -n "$SCRIPT" || failed_checks=$((failed_checks + 1))
local file
for file in "$SCRIPT" "$SRC_DIR"/lib/*.sh; do
bash -n "$file" || failed_checks=$((failed_checks + 1))
done
if command -v shellcheck >/dev/null 2>&1; then
shellcheck "$SCRIPT" "$TEST_DIR"/*.sh ||
# -x follows the source lines; the library files are named as well,
# because shellcheck only reports on the files it is given.
shellcheck -x "$SCRIPT" "$SRC_DIR"/lib/*.sh "$TEST_DIR"/*.sh ||
failed_checks=$((failed_checks + 1))
else
printf 'skipped: shellcheck is not installed\n'
fi
if command -v shfmt >/dev/null 2>&1; then
shfmt -i 2 -ci -d "$SCRIPT" "$TEST_DIR"/*.sh ||
shfmt -i 2 -ci -d "$SCRIPT" "$SRC_DIR"/lib/*.sh "$TEST_DIR"/*.sh ||
failed_checks=$((failed_checks + 1))
else
printf 'skipped: shfmt is not installed\n'
+475
View File
@@ -0,0 +1,475 @@
#!/usr/bin/env bash
# test-hosts.sh - tests for the GitHub and Gitea steps (MIL-002): preflight
# checks, the dry run, creating the repositories and the push mirror, reusing
# existing repositories and reporting a partial failure. A stub curl answers
# from a routes file and records every call; no real host is contacted.
# Sourced by run-tests.sh.
# shellcheck disable=SC2016 # snippet and fixture text is literal on purpose
run_dry() {
run_cli "$1" --config "$WORK/config.env" --env "$WORK/.env"
}
run_apply() {
run_cli "$1" --apply --config "$WORK/config.env" --env "$WORK/.env"
}
# position LINE: the number of the first recorded call equal to LINE.
position() {
calls | grep -n -x -F "$1" | head -n 1 | cut -d: -f1
}
readonly GITHUB_REPO_CALL="POST https://api.github.com/orgs/acme-org/repos"
readonly GITEA_REPO_CALL="POST https://git.example.test/api/v1/orgs/TirSystem/repos"
readonly MIRROR_CALL="POST https://git.example.test/api/v1/repos/TirSystem/my-app/push_mirrors"
readonly SYNC_CALL="POST https://git.example.test/api/v1/repos/TirSystem/my-app/push_mirrors-sync"
# ------------------------------------------------------------------ dry run
test_dry_run_prints_the_plan_and_only_reads() {
setup_hosts
run_dry "$ANSWERS_GITHUB"
assert_status "dry run" 0 "$STATUS"
assert_contains "Gitea plan" "$OUT" "Gitea repository : create (private) with the AGPL-3.0 license https://git.example.test/TirSystem/my-app"
assert_contains "GitHub plan" "$OUT" "GitHub repository : create (private), empty https://github.com/acme-org/my-app"
assert_contains "mirror plan" "$OUT" "Push mirror : Gitea -> GitHub every 10m0s"
assert_contains "origin plan" "$OUT" "Local origin : will use SSH (the SSH test passed)"
assert_contains "says it is a dry run" "$OUT" "Dry run: nothing was created. Run again with --apply"
assert_not_contains "no creation report" "$OUT" "This is what exists now"
assert_not_contains "only reads" "$(calls)" "POST"
assert_not_contains "never deletes" "$(calls)" "DELETE"
assert_not_contains "no PUT or PATCH" "$(calls)" "PATCH"
}
test_dry_run_does_not_ask_to_confirm() {
setup_hosts
run_dry "$ANSWERS_GITHUB"
assert_not_contains "no final question" "$ERR" "Create these now"
}
test_ssh_failure_means_https() {
setup_hosts
write_ssh_stub 255
run_dry "$ANSWERS_GITHUB"
assert_status "ssh failure is not fatal" 0 "$STATUS"
assert_contains "origin falls back to HTTPS" "$OUT" "will use HTTPS (SSH test: failed"
}
test_ssh_without_the_ssh_tool_is_not_fatal() {
setup_hosts
# A PATH that has the stubs and the basic tools but no ssh.
rm -f "$WORK/bin/ssh"
run_lib "" 'command() { if [[ $1 == -v && $2 == ssh ]]; then return 1; fi; builtin command "$@"; }
CONFIG[GITEA_URL]=https://git.example.test CONFIG[GITEA_SSH_PORT]=10022
test_gitea_ssh
echo "${STATE[is_ssh_ok]}|${STATE[ssh_note]}"'
assert_eq "no ssh installed" "0|not tested (ssh is not installed)" "$OUT"
}
test_ssh_uses_the_configured_port() {
setup_hosts
printf 'GITEA_SSH_PORT=2222\n' >>"$WORK/config.env"
run_dry "$ANSWERS_GITHUB"
assert_status "custom port" 0 "$STATUS"
assert_contains "port passed to ssh" "$(cat "$WORK/ssh.args")" "2222"
assert_contains "Gitea host" "$(cat "$WORK/ssh.args")" "git@git.example.test"
assert_contains "batch mode, no prompts" "$(cat "$WORK/ssh.args")" "BatchMode=yes"
assert_contains "unknown host keys are refused" "$(cat "$WORK/ssh.args")" "StrictHostKeyChecking=yes"
}
test_config_validates_ssh_port_and_interval() {
local key value expected
while IFS='|' read -r key value expected; do
printf 'GITEA_URL=https://git.example.test\n%s=%s\n' "$key" "$value" >"$WORK/c.env"
run_lib "" "parse_env_file \"$WORK/c.env\" CONFIG_KEYS CONFIG
validate_config"
assert_status "$key=$value" 1 "$STATUS"
assert_contains "$key=$value message" "$ERR" "$expected"
done <<'EOF'
GITEA_SSH_PORT|abc|port number
GITEA_SSH_PORT|0|port number
GITEA_SSH_PORT|70000|port number
MIRROR_INTERVAL|soon|10m0s or 8h0m0s
MIRROR_INTERVAL|10|10m0s or 8h0m0s
MIRROR_INTERVAL|10 m|10m0s or 8h0m0s
EOF
}
# -------------------------------------------------------------- preflight
test_gitea_token_rejected() {
setup_hosts
prepend_route 'GET|/api/v1/user|401|{"message":"token is required"}'
run_apply "$ANSWERS_GITHUB"
assert_status "bad Gitea token" 1 "$STATUS"
assert_contains "says what failed" "$ERR" "Gitea rejected the token: authentication failed"
assert_contains "shows the server's words" "$ERR" "token is required"
assert_not_contains "nothing created" "$(calls)" "POST"
assert_not_contains "no creation report before anything was created" "$OUT" "This is what exists now"
assert_not_contains "GitHub not contacted first" "$(calls)" "api.github.com"
}
test_github_token_rejected() {
setup_hosts
prepend_route 'GET|api.github.com/user|401|{"message":"Bad credentials"}'
run_apply "$ANSWERS_GITHUB"
assert_status "bad GitHub token" 1 "$STATUS"
assert_contains "says what failed" "$ERR" "GitHub rejected the token: authentication failed"
assert_not_contains "nothing created" "$(calls)" "POST"
}
test_gitea_owner_must_exist() {
setup_hosts
prepend_route 'GET|/api/v1/orgs/TirSystem|404|{"message":"not found"}'
run_apply "$ANSWERS_GITHUB"
assert_status "unknown Gitea owner" 1 "$STATUS"
assert_contains "message" "$ERR" "neither your account (gitea-user) nor an organization"
assert_not_contains "nothing created" "$(calls)" "POST"
}
test_gitea_organization_needs_create_permission() {
setup_hosts
prepend_route 'GET|/api/v1/users/gitea-user/orgs/TirSystem/permissions|200|{"can_create_repository":false}'
run_apply "$ANSWERS_GITHUB"
assert_status "no permission" 1 "$STATUS"
assert_contains "message" "$ERR" "you may not create repositories in the Gitea organization 'TirSystem'"
}
test_github_owner_must_be_a_member() {
setup_hosts
prepend_route 'GET|api.github.com/user/memberships/orgs/acme-org|404|{"message":"Not Found"}'
run_apply "$ANSWERS_GITHUB"
assert_status "not a member" 1 "$STATUS"
assert_contains "message" "$ERR" "neither your account (octo-user) nor an organization you belong to"
prepend_route 'GET|api.github.com/user/memberships/orgs/acme-org|200|{"state":"pending"}'
run_apply "$ANSWERS_GITHUB"
assert_status "pending membership" 1 "$STATUS"
assert_contains "pending message" "$ERR" "membership of the GitHub organization 'acme-org' is not active"
}
test_license_must_be_offered_when_github_is_chosen() {
setup_hosts
prepend_route 'GET|/api/v1/licenses|200|[{"key":"MIT","name":"MIT"}]'
run_apply "$ANSWERS_GITHUB"
assert_status "no AGPL" 1 "$STATUS"
assert_contains "message" "$ERR" "does not offer the AGPL-3.0 license"
assert_not_contains "nothing created" "$(calls)" "POST"
# Without GitHub no license is needed, so the same server is fine.
setup_hosts
prepend_route 'GET|/api/v1/licenses|200|[{"key":"MIT","name":"MIT"}]'
printf 'GITEA_TOKEN=%s\n' "$FAKE_GITEA_TOKEN" >"$WORK/.env"
run_dry "$ANSWERS_GITEA_ONLY"
assert_status "Gitea only" 0 "$STATUS"
}
test_existing_repository_with_content_stops_the_run() {
setup_hosts
prepend_route 'GET|/api/v1/repos/TirSystem/my-app|200|{"empty":false}'
prepend_route 'GET|/api/v1/repos/TirSystem/my-app/contents|200|[{"name":"README.md","type":"file"}]'
run_apply "$ANSWERS_GITHUB"
assert_status "repository with content" 1 "$STATUS"
assert_contains "message" "$ERR" "the Gitea repository TirSystem/my-app already exists and has content"
assert_not_contains "nothing created" "$(calls)" "POST"
}
test_network_failure_stops_before_creating() {
setup_hosts
prepend_route 'GET|/api/v1/user|exit7|'
run_apply "$ANSWERS_GITHUB"
assert_status "no connection" 1 "$STATUS"
assert_contains "message" "$ERR" "could not reach git.example.test: could not connect"
assert_not_contains "nothing created" "$(calls)" "POST"
}
test_token_for_another_github_account_is_reported() {
setup_hosts
prepend_route 'GET|api.github.com/user|200|{"login":"someone-else"}'
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "run continues" 0 "$STATUS"
assert_contains "warns" "$ERR" "GITHUB_USER is 'octo-user' but the token belongs to 'someone-else'"
assert_contains "mirror uses the account the token belongs to" "$(cat "$WORK/curl.bodies")" '"remote_username":"someone-else"'
}
# ------------------------------------------------------------ apply: success
test_apply_creates_repositories_and_mirror_in_order() {
setup_hosts
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "apply" 0 "$STATUS"
assert_before "GitHub before Gitea" "$(position "$GITHUB_REPO_CALL")" "$(position "$GITEA_REPO_CALL")"
assert_before "Gitea before the mirror" "$(position "$GITEA_REPO_CALL")" "$(position "$MIRROR_CALL")"
assert_before "mirror before the sync" "$(position "$MIRROR_CALL")" "$(position "$SYNC_CALL")"
assert_contains "final report" "$OUT" "Done. This is what exists now:"
assert_contains "GitHub created" "$OUT" "GitHub repository : created https://github.com/acme-org/my-app"
assert_contains "Gitea created" "$OUT" "Gitea repository : created https://git.example.test/TirSystem/my-app"
assert_contains "mirror created" "$OUT" "Push mirror : created Gitea -> https://github.com/acme-org/my-app.git"
assert_not_contains "never deletes" "$(calls)" "DELETE"
assert_eq "temporary files removed" "" "$(find "$WORK/tmp" -mindepth 1)"
}
test_apply_sends_the_right_request_bodies() {
setup_hosts
run_apply "$ANSWERS_GITHUB"$'y\n'
local bodies
bodies="$(cat "$WORK/curl.bodies")"
assert_contains "GitHub name" "$bodies" '"name":"my-app"'
assert_contains "GitHub is created empty" "$bodies" '"private":true,"auto_init":false}'
assert_contains "Gitea gets the license" "$bodies" '"license":"AGPL-3.0"'
assert_contains "Gitea is initialised with it" "$bodies" '"auto_init":true'
assert_contains "default branch" "$bodies" '"default_branch":"main"'
assert_contains "description" "$bodies" '"description":"A test app"'
assert_contains "mirror target without credentials" "$bodies" '"remote_address":"https://github.com/acme-org/my-app.git"'
assert_not_contains "no credentials in the address" "$bodies" 'https://octo-user:'
assert_not_contains "no credentials in the address (at sign)" "$bodies" '@github.com'
assert_contains "mirror account" "$bodies" '"remote_username":"octo-user"'
assert_contains "the token is the mirror password" "$bodies" "\"remote_password\":\"$FAKE_GITHUB_PAT\""
assert_contains "mirror interval" "$bodies" '"interval":"10m0s"'
assert_contains "push on commit asked for" "$bodies" '"sync_on_commit":true'
}
test_apply_never_prints_or_passes_a_token() {
setup_hosts
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_not_contains "no Gitea token in output" "$OUT$ERR" "$FAKE_GITEA_TOKEN"
assert_not_contains "no GitHub token in output" "$OUT$ERR" "$FAKE_GITHUB_PAT"
assert_not_contains "no token on a command line" "$(cat "$WORK/curl.args")" "$FAKE_GITEA_TOKEN"
assert_not_contains "no GitHub token on a command line" "$(cat "$WORK/curl.args")" "$FAKE_GITHUB_PAT"
assert_contains "Gitea token in the private config" "$(cat "$WORK/curl.config")" "Authorization: token $FAKE_GITEA_TOKEN"
assert_contains "GitHub token in the private config" "$(cat "$WORK/curl.config")" "Authorization: Bearer $FAKE_GITHUB_PAT"
}
test_apply_with_gitea_only() {
setup_hosts
printf 'GITEA_TOKEN=%s\n' "$FAKE_GITEA_TOKEN" >"$WORK/.env"
run_apply "$ANSWERS_GITEA_ONLY"$'y\n'
assert_status "Gitea only" 0 "$STATUS"
assert_contains "Gitea repository created" "$(calls)" "$GITEA_REPO_CALL"
assert_contains "created empty" "$(cat "$WORK/curl.bodies")" '"auto_init":false'
assert_not_contains "no license" "$(cat "$WORK/curl.bodies")" "license"
assert_not_contains "no GitHub call" "$(calls)" "api.github.com"
assert_not_contains "no mirror call" "$(calls)" "push_mirrors"
assert_contains "GitHub not used" "$OUT" "GitHub repository : not used"
assert_contains "mirror not used" "$OUT" "Push mirror : not used"
}
test_apply_declined_creates_nothing() {
setup_hosts
run_apply "$ANSWERS_GITHUB"$'n\n'
assert_status "answered no" 0 "$STATUS"
assert_contains "says so" "$OUT" "Nothing was created."
assert_not_contains "no POST" "$(calls)" "POST"
assert_not_contains "no report" "$OUT" "This is what exists now"
}
test_apply_for_user_owners_uses_the_user_endpoints() {
setup_hosts
write_routes <<'ROUTES'
GET|/api/v1/user|200|{"login":"gitea-user"}
GET|/api/v1/licenses|200|[{"key":"AGPL-3.0"}]
GET|/api/v1/repos/gitea-user/my-app|404|{"message":"not found"}
GET|api.github.com/user|200|{"login":"octo-user"}
GET|api.github.com/repos/octo-user/my-app|404|{"message":"Not Found"}
GET|/api/v1/repos/gitea-user/my-app/push_mirrors|200|[]
GET|/api/v1/repos/gitea-user/my-app/push_mirrors|200|[{"remote_address":"https://github.com/octo-user/my-app.git","sync_on_commit":true}]
POST|api.github.com/user/repos|201|{}
POST|/api/v1/user/repos|201|{}
POST|/api/v1/repos/gitea-user/my-app/push_mirrors|200|{}
POST|/api/v1/repos/gitea-user/my-app/push_mirrors-sync|200|{}
ROUTES
run_apply $'my-app\n\n\ngitea-user\ny\n\n\n\ny\n'
assert_status "user owners" 0 "$STATUS"
assert_contains "GitHub user endpoint" "$(calls)" "POST https://api.github.com/user/repos"
assert_contains "Gitea user endpoint" "$(calls)" "POST https://git.example.test/api/v1/user/repos"
assert_contains "mirror target of the GitHub account" "$(cat "$WORK/curl.bodies")" '"remote_address":"https://github.com/octo-user/my-app.git"'
assert_not_contains "no organization endpoint" "$(calls)" "/orgs/"
}
test_public_visibility_and_special_characters_in_the_description() {
setup_hosts
printf 'GITEA_TOKEN=%s\n' "$FAKE_GITEA_TOKEN" >"$WORK/.env"
run_apply $'my-app\nSay "hi" \\ there\npublic\nTirSystem\nn\n\nn\ny\n'
assert_status "apply" 0 "$STATUS"
assert_contains "public" "$(cat "$WORK/curl.bodies")" '"private":false'
assert_contains "description escaped" "$(cat "$WORK/curl.bodies")" '"description":"Say \"hi\" \\ there"'
}
test_mirror_interval_comes_from_the_configuration() {
setup_hosts
printf 'MIRROR_INTERVAL=1h0m0s\n' >>"$WORK/config.env"
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "apply" 0 "$STATUS"
assert_contains "interval sent" "$(cat "$WORK/curl.bodies")" '"interval":"1h0m0s"'
assert_contains "interval planned" "$OUT" "every 1h0m0s"
}
# ------------------------------------------------------------------- reuse
test_empty_github_repository_can_be_reused() {
setup_hosts
prepend_route 'GET|api.github.com/repos/acme-org/my-app|200|{"name":"my-app"}'
run_dry "$ANSWERS_GITHUB"
assert_contains "plan offers reuse" "$OUT" "reuse the existing empty repository (you will be asked to confirm)"
assert_not_contains "dry run does not ask" "$ERR" "Reuse it"
setup_hosts
prepend_route 'GET|api.github.com/repos/acme-org/my-app|200|{"name":"my-app"}'
run_apply "$ANSWERS_GITHUB"$'y\ny\n'
assert_status "reuse" 0 "$STATUS"
assert_contains "asked" "$ERR" "GitHub repository https://github.com/acme-org/my-app already exists"
assert_not_contains "GitHub repository not created again" "$(calls)" "$GITHUB_REPO_CALL"
assert_contains "reported as reused" "$OUT" "GitHub repository : reused https://github.com/acme-org/my-app"
assert_contains "the rest is created" "$(calls)" "$GITEA_REPO_CALL"
}
test_declining_the_reuse_stops_the_run() {
setup_hosts
prepend_route 'GET|api.github.com/repos/acme-org/my-app|200|{"name":"my-app"}'
run_apply "$ANSWERS_GITHUB"$'y\nn\n'
assert_status "reuse declined" 1 "$STATUS"
assert_contains "message" "$ERR" "stopped: choose another name or remove the existing repository"
assert_not_contains "nothing created" "$(calls)" "POST"
}
test_gitea_repository_with_only_the_license_can_be_reused() {
setup_hosts
prepend_route 'GET|/api/v1/repos/TirSystem/my-app|200|{"empty":false}'
prepend_route 'GET|/api/v1/repos/TirSystem/my-app/contents|200|[{"name":"LICENSE","type":"file","path":"LICENSE"}]'
run_apply "$ANSWERS_GITHUB"$'y\ny\n'
assert_status "license only" 0 "$STATUS"
assert_not_contains "Gitea repository not created again" "$(calls)" "$GITEA_REPO_CALL"
assert_eq "mirror created once" "1" "$(calls | grep -c -x -F "$MIRROR_CALL")"
assert_contains "reported" "$OUT" "Gitea repository : reused https://git.example.test/TirSystem/my-app"
# Without GitHub the license is not expected, so the repository has content.
setup_hosts
printf 'GITEA_TOKEN=%s\n' "$FAKE_GITEA_TOKEN" >"$WORK/.env"
prepend_route 'GET|/api/v1/repos/TirSystem/my-app|200|{"empty":false}'
prepend_route 'GET|/api/v1/repos/TirSystem/my-app/contents|200|[{"name":"LICENSE","type":"file"}]'
run_dry "$ANSWERS_GITEA_ONLY"
assert_status "license without GitHub" 1 "$STATUS"
assert_contains "has content" "$ERR" "already exists and has content"
}
test_reusing_an_empty_gitea_repository_warns_about_the_license() {
setup_hosts
prepend_route 'GET|/api/v1/repos/TirSystem/my-app|200|{"empty":true}'
run_apply "$ANSWERS_GITHUB"$'y\ny\n'
assert_status "reuse empty Gitea repository" 0 "$STATUS"
assert_contains "warning" "$ERR" "the AGPL-3.0 license is not added to it"
}
test_an_existing_mirror_is_reused() {
setup_hosts
prepend_route 'GET|/api/v1/repos/TirSystem/my-app/push_mirrors|200|[{"remote_address":"https://github.com/acme-org/my-app.git","sync_on_commit":true,"last_error":""}]'
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "mirror exists" 0 "$STATUS"
assert_eq "mirror not created twice" "" "$(position "$MIRROR_CALL")"
assert_contains "first sync still requested" "$(calls)" "$SYNC_CALL"
assert_contains "reported" "$OUT" "Push mirror : reused Gitea -> https://github.com/acme-org/my-app.git"
}
# ----------------------------------------------------------------- failures
test_partial_failure_reports_what_exists() {
setup_hosts
prepend_route 'POST|/api/v1/orgs/TirSystem/repos|422|{"message":"repository already exists"}'
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "Gitea creation fails" 1 "$STATUS"
assert_contains "reason" "$ERR" "cannot create the Gitea repository: rejected"
assert_contains "server's words" "$ERR" "repository already exists"
assert_contains "report header" "$OUT" "The run stopped before it finished. This is what exists now:"
assert_contains "GitHub was created" "$OUT" "GitHub repository : created https://github.com/acme-org/my-app"
assert_contains "Gitea failed" "$OUT" "Gitea repository : FAILED"
assert_contains "mirror not attempted" "$OUT" "Push mirror : not attempted"
assert_contains "how to continue" "$OUT" "To continue: fix the problem named above and run the same command again with --apply."
assert_contains "nothing deleted" "$OUT" "Nothing is deleted automatically."
assert_not_contains "never deletes" "$(calls)" "DELETE"
assert_not_contains "no mirror call" "$(calls)" "push_mirrors"
assert_eq "temporary files removed" "" "$(find "$WORK/tmp" -mindepth 1)"
}
test_failure_of_the_first_step_leaves_the_rest_not_attempted() {
setup_hosts
prepend_route 'POST|api.github.com/orgs/acme-org/repos|403|{"message":"Resource not accessible by personal access token"}'
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "GitHub creation fails" 1 "$STATUS"
assert_contains "reason" "$ERR" "cannot create the GitHub repository: the token is valid but not allowed to do this"
assert_contains "GitHub failed" "$OUT" "GitHub repository : FAILED"
assert_contains "Gitea not attempted" "$OUT" "Gitea repository : not attempted"
assert_not_contains "no Gitea call" "$(calls)" "$GITEA_REPO_CALL"
}
test_mirror_failure_keeps_the_repositories_and_says_so() {
setup_hosts
prepend_route 'POST|/api/v1/repos/TirSystem/my-app/push_mirrors|403|{"message":"push mirrors are disabled"}'
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "mirror fails" 1 "$STATUS"
assert_contains "reason" "$ERR" "cannot create the push mirror"
assert_contains "server's words" "$ERR" "push mirrors are disabled"
assert_contains "GitHub kept" "$OUT" "GitHub repository : created"
assert_contains "Gitea kept" "$OUT" "Gitea repository : created"
assert_contains "mirror failed" "$OUT" "Push mirror : FAILED"
}
test_sync_on_commit_not_applied_is_reported() {
setup_hosts
sed -i 's/"sync_on_commit":true,"interval"/"sync_on_commit":false,"interval"/' "$WORK/routes"
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "still succeeds" 0 "$STATUS"
assert_contains "warning" "$ERR" "Gitea did not apply sync_on_commit (a known server issue)"
assert_contains "report mentions the interval" "$OUT" "(syncs every 10m0s, not on every commit)"
}
test_first_sync_error_is_reported() {
setup_hosts
sed -i 's/"last_error":""/"last_error":"push failed: authentication required"/' "$WORK/routes"
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "still succeeds" 0 "$STATUS"
assert_contains "warning" "$ERR" "the first mirror sync reported: push failed: authentication required"
}
test_first_sync_request_failure_is_only_a_warning() {
setup_hosts
prepend_route 'POST|/api/v1/repos/TirSystem/my-app/push_mirrors-sync|500|{"message":"boom"}'
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "still succeeds" 0 "$STATUS"
assert_contains "warning" "$ERR" "could not ask Gitea for the first sync (HTTP 500)"
}
test_a_token_in_a_server_message_is_redacted() {
setup_hosts
prepend_route "POST|/api/v1/orgs/TirSystem/repos|422|{\"message\":\"bad credentials $FAKE_GITHUB_PAT given\"}"
run_apply "$ANSWERS_GITHUB"$'y\n'
assert_status "Gitea creation fails" 1 "$STATUS"
assert_not_contains "token redacted" "$OUT$ERR" "$FAKE_GITHUB_PAT"
assert_contains "redaction visible" "$ERR" "[redacted]"
}
# ------------------------------------------------------------ JSON helpers
test_json_has_value_and_values() {
printf '[{"key": "AGPL-3.0","name":"a"},{"key":"MIT","name":"b"}]\n' >"$WORK/l.json"
run_lib "" "json_has_value '$WORK/l.json' key AGPL-3.0 && echo yes1
json_has_value '$WORK/l.json' key MIT && echo yes2
json_has_value '$WORK/l.json' key GPL-3.0 || echo no3
json_values '$WORK/l.json' name"
assert_eq "pairs and values" $'yes1\nyes2\nno3\na\nb' "$OUT"
}
test_mirror_field_with_and_without_jq() {
local mode
printf '[{"remote_address":"https://github.com/o/a.git","sync_on_commit":false,"last_error":"x"},{"remote_address":"https://github.com/o/b.git","sync_on_commit":true,"last_error":""}]\n' >"$WORK/m.json"
for mode in 0 1; do
if ((mode)) && ! command -v jq >/dev/null 2>&1; then
continue
fi
run_lib "" "HAS_JQ=$mode
mirror_field '$WORK/m.json' https://github.com/o/a.git sync_on_commit
mirror_field '$WORK/m.json' https://github.com/o/a.git last_error"
assert_eq "first mirror (HAS_JQ=$mode)" $'false\nx' "$OUT"
done
if command -v jq >/dev/null 2>&1; then
run_lib "" "HAS_JQ=1
mirror_field '$WORK/m.json' https://github.com/o/b.git sync_on_commit"
assert_eq "the right mirror is chosen with jq" "true" "$OUT"
fi
}
+31 -18
View File
@@ -4,43 +4,47 @@
# (MIL-001 Go/No-Go criteria 2 to 4). Sourced by run-tests.sh.
# shellcheck disable=SC2016 # snippet and fixture text is literal on purpose
readonly ANSWERS_GITHUB=$'my-app\nA test app\n\nTirSystem\ny\nacme-org\n\nn\n'
readonly ANSWERS_GITEA_ONLY=$'my-app\n\n\nTirSystem\nn\n\nn\n'
# listing: all files under the work directory except the test's own captures.
listing() {
find "$WORK" -type f ! -name out.txt ! -name err.txt ! -name snippet.sh \
! -name 'curl.*' | sort
! -name 'curl.*' ! -name 'routes*' ! -name 'ssh.args' | sort
}
test_full_run_with_github() {
write_fixtures
write_curl_stub
setup_hosts
local before after
before="$(listing)"
run_cli "$ANSWERS_GITHUB" --config "$WORK/config.env" --env "$WORK/.env"
after="$(listing)"
assert_status "full run" 0 "$STATUS"
assert_contains "summary" "$OUT" "nothing has been created yet"
assert_contains "dry run" "$OUT" "Dry run: nothing was created"
assert_contains "plan" "$OUT" "create (private) with the AGPL-3.0 license"
assert_contains "Gitea link derived from config" "$OUT" "https://git.example.test/TirSystem/my-app"
assert_contains "GitHub link uses the chosen organization" "$OUT" "https://github.com/acme-org/my-app"
assert_contains "AGPL noted" "$OUT" "AGPL license applied"
assert_contains "credential state" "$OUT" "GITEA_TOKEN set, GITHUB_PAT set"
assert_not_contains "no Gitea token in output" "$OUT$ERR" "$FAKE_GITEA_TOKEN"
assert_not_contains "no GitHub token in output" "$OUT$ERR" "$FAKE_GITHUB_PAT"
assert_file_missing "no network call" "$WORK/curl.args"
assert_contains "reads from Gitea" "$(calls)" "GET https://git.example.test/api/v1/user"
assert_contains "reads from GitHub" "$(calls)" "GET https://api.github.com/user"
assert_not_contains "a dry run only reads" "$(calls)" "POST"
assert_not_contains "a dry run never deletes" "$(calls)" "DELETE"
assert_eq "no file created or changed" "$before" "$after"
assert_eq "temporary files removed" "" "$(find "$WORK/tmp" -mindepth 1)"
}
test_full_run_without_github() {
write_fixtures
setup_hosts
printf 'GITEA_TOKEN=%s\n' "$FAKE_GITEA_TOKEN" >"$WORK/.env"
run_cli "$ANSWERS_GITEA_ONLY" --config "$WORK/config.env" --env "$WORK/.env"
assert_status "Gitea-only run needs no GitHub credentials" 0 "$STATUS"
assert_contains "GitHub not used" "$OUT" "GitHub : not used"
assert_not_contains "no AGPL line" "$OUT" "AGPL"
assert_contains "GITHUB_PAT not set" "$OUT" "GITHUB_PAT not set"
assert_not_contains "no call to GitHub" "$(calls)" "api.github.com"
assert_not_contains "no license lookup without GitHub" "$(calls)" "/licenses"
assert_contains "plan says GitHub is not used" "$OUT" "GitHub repository : not used"
}
test_github_chosen_without_credentials_fails() {
@@ -99,11 +103,11 @@ test_usage_errors() {
assert_contains "help shows exit codes" "$OUT" "Exit codes"
run_cli "" --version
assert_status "version" 0 "$STATUS"
assert_contains "version output" "$OUT" "RepoFoundry 0.1.0"
assert_contains "version output" "$OUT" "RepoFoundry 0.2.0"
}
test_warns_when_env_is_not_ignored_by_git() {
write_fixtures
setup_hosts
git init -q "$WORK/repo"
cp "$WORK/.env" "$WORK/repo/.env"
cp "$WORK/config.env" "$WORK/repo/config.env"
@@ -117,12 +121,20 @@ test_warns_when_env_is_not_ignored_by_git() {
test_script_uses_no_unsafe_constructs() {
local code
code="$(grep -vE '^[[:space:]]*#' "$SCRIPT")"
code="$(cat "$SCRIPT" "$SRC_DIR"/lib/*.sh | grep -vE '^[[:space:]]*#')"
assert_not_contains "no rm -rf" "$code" "rm -rf"
assert_not_contains "no rm -r" "$code" "rm -r "
assert_not_contains "no eval" "$code" "eval "
assert_not_contains "no source" "$code" "source "
assert_not_contains "no dot-source" "$code" $'\n. '
# Only the script's own library files are sourced, by a fixed path; a
# configuration file never is.
local line
while IFS= read -r line; do
check
if [[ $line != 'source "$SCRIPT_DIR/lib/'*'.sh"' ]]; then
fail "unexpected source line: $line"
fi
done < <(grep -hE '(^|[[:space:];])source ' <<<"$code")
check
if grep -Eq '^[[:space:]]*set -[A-Za-z]*x' <<<"$code"; then
fail "the script turns tracing on"
@@ -133,9 +145,9 @@ test_script_uses_no_unsafe_constructs() {
test_tracing_does_not_leak_secrets() {
# bash -x would print every assignment and command, secrets included, so
# the script switches tracing off and says so.
write_fixtures
setup_hosts
STATUS=0
PATH="$WORK/bin:$PATH" TMPDIR="$WORK/tmp" "$BASH" -x "$SCRIPT" \
PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" TMPDIR="$WORK/tmp" "$BASH" -x "$SCRIPT" \
--config "$WORK/config.env" --env "$WORK/.env" <<<"$ANSWERS_GITHUB" \
>"$WORK/out.txt" 2>"$WORK/err.txt" || STATUS=$?
assert_status "run under bash -x" 0 "$STATUS"
@@ -157,7 +169,7 @@ test_termination_removes_temp_files() {
if ! mkfifo "$WORK/in" 2>/dev/null; then
return 0
fi
PATH="$WORK/bin:$PATH" TMPDIR="$WORK/tmp" "$BASH" "$SCRIPT" \
PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" TMPDIR="$WORK/tmp" "$BASH" "$SCRIPT" \
--config "$WORK/config.env" --env "$WORK/.env" <"$WORK/in" \
>/dev/null 2>&1 &
local pid=$! tries=0
@@ -182,19 +194,20 @@ test_script_lives_in_src() {
test_default_files_are_in_the_project_root() {
# A project copy: script in src/, config.env and .env one level up.
write_fixtures
setup_hosts
mkdir -p "$WORK/project/src"
cp "$SCRIPT" "$WORK/project/src/create-project.sh"
cp -R "$SRC_DIR/lib" "$WORK/project/src/lib"
cp "$WORK/config.env" "$WORK/project/config.env"
cp "$WORK/.env" "$WORK/project/.env"
STATUS=0
PATH="$WORK/bin:$PATH" TMPDIR="$WORK/tmp" "$BASH" "$WORK/project/src/create-project.sh" \
PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" TMPDIR="$WORK/tmp" "$BASH" "$WORK/project/src/create-project.sh" \
<<<"$ANSWERS_GITHUB" >"$WORK/out.txt" 2>"$WORK/err.txt" || STATUS=$?
assert_status "run with the default files" 0 "$STATUS"
assert_contains "found config.env in the project root" "$(cat "$WORK/out.txt")" "https://git.example.test/TirSystem/my-app"
# Run from another directory: the defaults follow the script, not the cwd.
STATUS=0
(cd "$WORK" && PATH="$WORK/bin:$PATH" TMPDIR="$WORK/tmp" "$BASH" "$WORK/project/src/create-project.sh" \
(cd "$WORK" && PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" TMPDIR="$WORK/tmp" "$BASH" "$WORK/project/src/create-project.sh" \
<<<"$ANSWERS_GITEA_ONLY" >"$WORK/out.txt" 2>"$WORK/err.txt") || STATUS=$?
assert_status "run from another directory" 0 "$STATUS"
}
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env bash
# test-structure.sh - guards for the split of create-project.sh into files
# with one responsibility each. Sourced by run-tests.sh.
# shellcheck disable=SC2016 # snippet and pattern text is literal on purpose
# lib_names: the names of the files in src/lib, one per line.
lib_names() {
local file
for file in "$SRC_DIR"/lib/*.sh; do
file="${file##*/}"
printf '%s\n' "${file%.sh}"
done
}
test_every_library_file_is_loaded_and_nothing_else() {
local loaded on_disk
loaded="$(grep -E '^source "\$SCRIPT_DIR/lib/' "$SCRIPT" | sed -e 's|.*/lib/||' -e 's|\.sh"$||' | sort)"
on_disk="$(lib_names | sort)"
assert_eq "the files that are loaded are the files in src/lib" "$on_disk" "$loaded"
}
test_every_library_file_states_one_responsibility() {
local name line
for name in $(lib_names); do
line="$(head -n 4 "$SRC_DIR/lib/$name.sh" | grep -m 1 "^# $name.sh - " || true)"
check
if [[ -z $line ]]; then
fail "src/lib/$name.sh must name its responsibility in its first lines ('# $name.sh - ...')"
fi
check
if ! grep -q '^# Provides: ' "$SRC_DIR/lib/$name.sh" && [[ $name != constants ]]; then
fail "src/lib/$name.sh does not list what it provides"
fi
done
}
test_no_function_is_defined_twice() {
local duplicates
duplicates="$(cat "$SCRIPT" "$SRC_DIR"/lib/*.sh | grep -oE '^[a-z_]+\(\) \{' | sort | uniq -d)"
assert_eq "each function is defined in exactly one file" "" "$duplicates"
}
test_the_listed_functions_exist_in_their_file() {
local name list fn
for name in $(lib_names); do
list="$(sed -n 's/^# Provides: //p' "$SRC_DIR/lib/$name.sh" | tr -d ',')"
for fn in $list; do
check
if ! grep -q "^$fn() {" "$SRC_DIR/lib/$name.sh"; then
fail "src/lib/$name.sh says it provides $fn but does not define it"
fi
done
done
}
test_library_files_have_no_side_effects_when_sourced_alone() {
# A library file only defines functions and constants: running it by itself
# (after the constants) prints nothing and makes no request.
local name
write_curl_stub
for name in $(lib_names); do
if [[ $name == constants ]]; then
continue
fi
STATUS=0
PATH="$WORK/bin:$PATH" STUB_DIR="$WORK" "$BASH" -c \
'SCRIPT_DIR="$1"; PROJECT_ROOT="$2"; source "$1/lib/constants.sh"; source "$1/lib/$3.sh"' \
_ "$SRC_DIR" "$REPO_ROOT" "$name" >"$WORK/out.txt" 2>"$WORK/err.txt" || STATUS=$?
assert_status "sourcing lib/$name.sh" 0 "$STATUS"
assert_eq "lib/$name.sh prints nothing" "" "$(cat "$WORK/out.txt" "$WORK/err.txt")"
done
assert_eq "no request made by sourcing" "" "$(calls)"
}
test_the_entry_point_is_small_and_holds_no_helpers() {
local helpers
helpers="$(grep -oE '^[a-z_]+\(\) \{' "$SCRIPT" | tr -d '(){ ' | sort | tr '\n' ' ')"
assert_eq "only finish and main live in the entry point" "finish main " "$helpers"
}
test_no_library_file_is_ignored_by_git() {
# The Python template in .gitignore ignores any folder named lib/; a file
# that git ignores would silently be left out of every commit.
local name
for name in $(lib_names); do
check
if git -C "$REPO_ROOT" check-ignore -q --no-index "src/lib/$name.sh"; then
fail "src/lib/$name.sh is ignored by git; check .gitignore (!src/lib)"
fi
done
}