Translation System¶

Overview¶

eXeLearning uses XLF (XLIFF) files for internationalization. Translation files are stored in the translations/ directory and loaded at server startup.

Supported Languages¶

The application supports these interface languages:

Code	Language
`en`	English (default)
`es`	Español
`ca`	Català
`va`	Valencià
`eu`	Euskara
`fr`	Français
`gl`	Galego
`pt`	Português
`eo`	Esperanto
`ro`	Română

Additional locales are available for exported content packages (see src/services/translation.ts for the full list).

Translation Files¶

Translations are stored as XLF files in translations/:

translations/
├── messages.en.xlf
├── messages.es.xlf
├── messages.ca.xlf
├── messages.eu.xlf
├── messages.fr.xlf
├── messages.gl.xlf
├── messages.pt.xlf
├── messages.eo.xlf
├── messages.ro.xlf
└── messages.va.xlf

Using Translations¶

In TypeScript (Backend)¶

import { trans } from '../services/translation';

// Simple translation
const message = trans('welcome.message');

// With parameters
const greeting = trans('hello.user', { name: 'John' });
// Parameters support both %param% and {param} formats

In Nunjucks Templates¶

{{ trans('page.title') }}
{{ trans('welcome.user', { name: user.name }) }}

In JavaScript (Frontend)¶

// GUI translations (workarea UI strings)
const label = _('Export page');

// Content translations (iDevice content strings)
const text = c_('Page title');

// Template literals are also supported
const html = `<span>${_('Save')}</span>`;

Generated Sources: Build Before Extracting¶

The extraction scans the working tree, not the dependency graph. Some of the strings it must find no longer live in files the repository commits: they belong to libraries vendored out of a pinned npm package into a scanned path by the build, and left gitignored. public/app/common/edicuatex/ is the first one — scripts/vendor-edicuatex.ts writes it as the first step of build:all — and it will not be the last, because the project is steadily moving third-party code from hand-maintained copies to pinned packages.

This matters because a checkout that has not been built looks completely normal to the scanner. Every other source scans fine; the key set simply comes out short. That is harmless while only adding keys, and destructive under --remove-obsolete, which treats the extracted set as the whole truth and deletes every trans-unit outside it from every locale — around 675 of them, in every language, for edicuatex alone.

A tree that is present can be just as short. An interrupted vendor run leaves a directory that exists and holds some of its files, and the strings in the files it never wrote are exactly as invisible to the scan as if nothing were there at all — with the difference that an existence check reports everything is fine. A tree can also be stale: bump the dependency and run bun install without make vendor-edicuatex, and every file is still there with the previous version's contents, so the strings only the new one carries look obsolete. Both count as incomplete.

Three things keep that from happening:

make translations, make translations-cleanup and make translations-sort depend on vendor-edicuatex, so the tree is regenerated before anything reads it. Running through make is always safe.
The commands warn when a registered tree is absent or incomplete, and --remove-obsolete refuses to run at all. This is what protects a direct bun cli translations … invocation, which bypasses make. The refusal can be overridden with --allow-missing-generated, which is destructive by design — use it only when you know the missing tree holds no strings.
GENERATED_SOURCE_DIRS in src/cli/commands/translations.ts is the registry the warning and the refusal read. Each entry may carry an inspect hook that compares the tree on disk against what its generator would write; edicuatex uses scripts/vendor-edicuatex.ts's own plan, the same knowledge behind vendor-edicuatex.ts --check, rather than a second list that could disagree with it — files that are missing and files whose contents differ, since either hides strings. Without a hook, an entry falls back to the existence check. A hook whose package is absent altogether reports nothing: there is no pinned version to compare against, and --extract-only and translations:sort delete nothing, so they must not start depending on node_modules. A package that is half-installed is the opposite case — the vendored tree is then whatever the previous version left behind, and nothing can say whether it still matches — so it is reported as incomplete: the two non-destructive commands still only warn, while --remove-obsolete refuses.

When you vendor a new library¶

If a library carrying translatable strings starts being vendored from an npm dependency into a scanned path, add it to GENERATED_SOURCE_DIRS in the same PR, give the entry an inspect hook built on the vendoring script's own file plan, and add the vendor-* target to the three translation targets in the Makefile. Nothing detects the omission automatically: the symptom is the silent one above, and it surfaces as strings disappearing from every locale weeks later.

A vendored tree does not belong in the registry when either is true:

It is committed to git rather than gitignored, so a checkout always has it — public/app/common/exe_math/ (MathJax), by the decision in ADR-2259-01.
It is excluded from scanning in EXCLUDE_FILE_PATTERNS, so its strings were never extracted in the first place.

Translation Commands¶

Extract New Translation Keys¶

Scan source files for translation function calls and add new keys to XLF files:

# Extract keys for all locales
make translations

# Extract for a specific locale
make translations LOCALE=es

# Only extract (no cleanup) — equivalent to make translations
bun cli translations --extract-only

Clean and Remove Obsolete Keys¶

Remove entries that no longer exist in the source code (destructive — irreversible without git). Read Generated Sources first: run through make, or this command deletes every string that lives only in a tree the build generates.

# Clean all locales
make translations-cleanup

# Clean a specific locale
make translations-cleanup LOCALE=es

# Equivalent CLI command
bun cli translations --clean-only --remove-obsolete

Sort Trans-Unit Order¶

Reorder <trans-unit> elements in all XLF files so they follow the same order as messages.en.xlf. Before sorting, the command verifies that messages.en.xlf is in sync with the source code and exits with a list of differences if it is not.

# Sort all locales
make translations-sort

# Sort a specific locale
make translations-sort LOCALE=es

XML comments inside <body> (e.g. ) are discarded during sorting, as they would be out of context after reordering.

Format XLF Files¶

Normalise <target> content and fix indentation in all XLF files:

Wraps <target> content in <![CDATA[...]]> when it contains characters that are invalid as raw XML (bare <, or & not followed by a predefined entity reference such as &, <, >, ", ').
Already-wrapped CDATA sections and valid plain-text targets are left untouched.
Normalises indentation: 6 spaces before <trans-unit>, 8 spaces before <source> and <target>.
Skips messages.en.xlf by default (English is the source language; its <target> entries mirror <source> and never need CDATA normalisation).

# Format all locales (except "en")
make translations-format

# Format a specific locale
make translations-format LOCALE=es

# Equivalent CLI command
bun cli translations:format
bun cli translations:format --locale=es

Other CLI Options¶

# Extract + clean in one pass (no removal of obsolete keys)
bun cli translations

# Only clean formatting/invalid entries (no removal)
bun cli translations --clean-only

# Process a specific locale
bun cli translations --locale=es --extract-only

# Force removal despite a missing generated tree (destructive; see Generated Sources)
bun cli translations --clean-only --remove-obsolete --allow-missing-generated

Recommended Command Order¶

When doing a full translation maintenance cycle, run the commands in this order:

# 1. Remove obsolete keys (strings removed from the source code)
make translations-cleanup

# 2. Extract new keys from source code into all XLF files
make translations

# 3. Sort all XLF files to match the canonical order of messages.en.xlf
make translations-sort

# 4. Wrap any <target> values that need CDATA and normalise indentation
make translations-format

All four commands accept an optional LOCALE=xx argument to restrict the operation to a single language.

Extraction Sources¶

The extractor scans these directories and file types:

Directory	Extensions	Patterns detected
`src/`	`*.ts`	`trans('key')`, `${TRANS_PREFIX}Key`
`views/`	`*.njk`	`'key' \\| trans`
`public/app/`	`*.js`	`_('key')`, `c_('key')`
`public/libs/`	`*.js`	`_('key')`, `c_('key')`
`public/files/perm/idevices/`	`*.js`	`_('key')`, `c_('key')`

Controlling What Gets Extracted¶

The extractor has three mechanisms to exclude strings, all in src/cli/commands/translations.ts.

`EXCLUDE_FILE_PATTERNS` — skip entire files or directories¶

Regex patterns matched against the full file path. Any file whose path matches is skipped entirely:

const EXCLUDE_FILE_PATTERNS = [
    /\.spec\.ts$/,           // Backend test files
    /\.test\.js$/,           // Frontend test files
    /[\\/]+exe_math[\\/]+/,  // MathJax (has its own _() calls)
    /[\\/]+node_modules[\\/]+/,
    // Admin panel excluded — see "Admin Panel" section below
    /[\\/]+views[\\/]+admin[\\/]+/,
    /[\\/]+app[\\/]+admin[\\/]+/,
    /[\\/]+routes[\\/]+admin/,
];

To exclude a new directory, add a regex entry here.

`EXCLUDE_EXACT_KEYS` — skip specific strings by exact value¶

A Set<string> of exact key values to ignore. Use this when a file contains strings that look like translation calls but are not UI labels (e.g. math expressions, code examples):

const EXCLUDE_EXACT_KEYS = new Set([
    'P + \\\\tfrac12 \\\\rho v^2 + \\\\rho g h = \\\\text{constant}',
    // ^ Bernoulli equation example in edicuatex lang file
]);

Note the double escaping: each \\ in the source file on disk becomes \\\\ in a TypeScript string literal (since the extractor reads raw file text, not evaluated JS values).

`INVALID_KEY_PATTERNS` — skip keys matching a pattern¶

Regex patterns matched against extracted key values. Keys matching any pattern are silently discarded. Used to filter out test fixture strings and documentation examples that accidentally match translation patterns:

const INVALID_KEY_PATTERNS = [
    /^test\./,         // test.key, test.something
    /^pattern\./,      // pattern.trans, pattern.t
    /^nonexistent\./,  // nonexistent.translation.key
    /^key$/,           // just "key"
    // ...
];

Admin Panel¶

The admin panel (/admin) is always displayed in English, regardless of the user's locale. This is intentional: the admin interface targets technical users and keeping it in a single language simplifies maintenance.

How it works¶

Translations for the admin panel are built by buildAdminTranslations(locale) in src/routes/admin.ts. The call site in src/routes/pages.ts hardcodes 'en' as the locale:

// src/routes/pages.ts
const t = buildAdminTranslations('en'); // Admin panel is English-only

The admin source files (views/admin/, public/app/admin/, src/routes/admin*) are also excluded from the main translation scanner via EXCLUDE_FILE_PATTERNS, so their strings never appear in the XLF files.

Enabling translations for the admin panel¶

If you want to translate the admin panel into other languages:

src/routes/pages.ts — replace 'en' with the locale variable:

const t = buildAdminTranslations(locale); // re-enabled translations

src/cli/commands/translations.ts — comment out the three admin exclusions in EXCLUDE_FILE_PATTERNS (currently lines 43–45):

// Comment the following 3 lines to scan the admin panel
// /[\\/]+views[\\/]+admin[\\/]+/,
// /[\\/]+app[\\/]+admin[\\/]+/,
// /[\\/]+routes[\\/]+admin/,

Run the normal extraction workflow to populate the XLF files:

make translations

Translate the new <target> entries in each translations/messages.*.xlf file.

Adding a New Language¶

Add the locale to LOCALES in src/services/translation.ts:

export const LOCALES: Record<string, string> = {
    en: 'English',
    es: 'Español',
    fr: 'Français',  // New language
    // ...
};

Create the XLF file:

cp translations/messages.en.xlf translations/messages.fr.xlf

Update target-language in the new XLF file's <file> element, then translate the <target> entries.
Run extraction to add any missing keys:

make translations LOCALE=fr

XLF File Format¶

<?xml version="1.0" encoding="UTF-8"?>
<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
  <file source-language="en" target-language="es" datatype="plaintext">
    <body>
      <trans-unit id="abc123" resname="welcome.message">
        <source>Welcome to eXeLearning</source>
        <target>Bienvenido a eXeLearning</target>
      </trans-unit>
    </body>
  </file>
</xliff>

resname — the key as it appears in source code (XML-escaped if it contains &, <, >, ")
<source> — the English string
<target> — the translated string; empty means untranslated

Locale Detection¶

The server detects the user's locale from:

User preference (stored in session/profile)
Accept-Language HTTP header
Default locale (en)

import { detectLocaleFromHeader, setLocale } from '../services/translation';

const locale = detectLocaleFromHeader(request.headers.get('accept-language'));
setLocale(locale);

Best Practices¶

Use natural-language strings as keys (_('Export page')) rather than dot-notation keys (_('menu.export.page')); this is the established pattern in this codebase.
Run make translations after adding any new translatable strings.
Run make translations-cleanup periodically to remove keys that no longer exist in the source.
Run make translations-sort to keep all XLF files consistently ordered (makes diffs easier to review).
Run make translations-format to wrap any <target> values that require CDATA and normalise indentation.
Never hardcode UI strings — always wrap them in _(), c_(), or trans().