Skip to main content
Notes 🐧💡💥
“If you can't find it, stash it better.” — Tux
  • Login

Portable by Design: A Modern Architecture for Sh1re Services

Sun, 11/09/2025 - 11:57am by alar

 Sh1re Architecture — Portable by Design

Access        → Sh1re
Flow          → TransferDepot
Compatibility → TLS Translate
Tools         → OPS Toolkit
Insight       → TD Detect

Core Message

Portable by Design
Everything in the Sh1re ecosystem is modular, self-contained, and runs cleanly on-prem today — while remaining ready for cloud or hybrid deployment tomorrow.

We are not locked into any platform.
We choose where things run.


🟦 The Sh1re (Reverse Proxy Gateway)

The Sh1re is the centralized access layer for internal services.

  • Consistent URLs for all applications
  • TLS termination and secure routing
  • Shields legacy systems from direct exposure
  • No changes required to existing services

Result:
A stable, portable front door that works the same on bare metal, VMs, or cloud load balancers.


🟩 TransferDepot (File Flow Service)

TransferDepot standardizes file movement between restricted and open environments.

  • Replaces ad-hoc scripts with a consistent workflow
  • Works with legacy systems (“Ye Olde Boxes”)
  • Simple, container-friendly architecture
  • No dependency on modern client capabilities

Result:
Reliable, repeatable file handling across environments.


🟧 TLS Translate (Modernization Layer)

TLS Translate bridges old and new security standards.

  • Accepts legacy TLS connections
  • Reissues using modern encryption
  • Transparent to both client and server

Result:
Legacy systems remain operational while meeting modern security expectations.


🟨 OPS Toolkit (Portable Utilities)

A collection of lightweight operational tools.

  • Self-contained Python components
  • Run on any Linux host
  • Can be scheduled, scripted, or containerized

Result:
Fast, flexible operational capability without heavy infrastructure.


🟪 TD Detect (Behavioral Insight Layer)

🟪 TD Detect — From Capability to Reality

One-Line Executive Statement

We’ve built a system that automatically detects abnormal file activity, system misuse, and unexpected data movement inside TransferDepot—even in our air-gapped environment—without anyone manually reviewing logs.


What That Actually Means (Backed by Today’s Work)

This isn’t a claim. It’s now implemented and proven.


🧱 Deterministic + Behavioral Detection

We parse logs once into structured events and run multiple detectors against a single source of truth.

  • Burst activity (rapid uploads)
  • File reuse and loops
  • Cross-group movement
  • User spread and access patterns
  • Size anomalies and sequence issues

👉 Result:
We detect known operational and misuse patterns automatically.


🧠 Vector-Based Anomaly Detection (Now Operational)

We added a second layer:

  • Embedding-based similarity (MiniLM + FAISS)
  • Detects content that does not match expected system behavior
  • Surfaces “foreign” or out-of-place entries

Validated with:

  • human text in logs
  • injected content (e.g., private key patterns)
  • malformed or non-log entries

👉 Result:
We detect unknown or unexpected behavior, not just predefined rules.


🔌 Portable, Environment-Independent Execution

  • TD_PATH allows scanning any dataset
  • Works on:
    • dev (Camelot)
    • sh0re / sh1re
    • offline laptop environments

👉 Result:
Same detector, same logic, anywhere.


🧪 Verified Test Harness

We built a controlled dataset that:

  • Exercises every rule-based detector
  • Forces vector anomalies across thresholds
  • Validates expected outputs end-to-end

👉 Result:
This is not experimental—it is testable and repeatable.


📡 Observable and Trustworthy Output

  • Clear alert messages (rule-based + vector)
  • Distance scoring visible
  • Alert artifacts written to disk
  • Explicit “no anomalies detected” state

👉 Result:
Operators can trust both alerts and silence


🚂 Fully Air-Gapped Operation

  • Models cached locally
  • No external calls required
  • FAISS + embeddings verified offline

👉 Result:
Advanced detection capability in restricted environments


🧭 What We Have Now

Not a script.

[ Logs ]
   ↓
[ Structured Events ]
   ↓
[ Rule Engine ]      → detects known patterns
   ↓
[ Vector Engine ]    → detects unknown anomalies
   ↓
[ Alerts + Artifacts ]

👉 This is a behavioral detection pipeline


🔥 The Shift (This Is the Real Story)

Before:

“We could analyze logs if needed”

After:

“We automatically detect and classify behavior in TransferDepot”


💼 Why This Matters (Management View)

This delivers:

1) Early Warning

  • Detects issues before users report them
  • Identifies broken workflows (loops, bursts)

2) Data Movement Visibility

  • Tracks files across zones (TTCS → ODSP → SHIRE)
  • Surfaces unexpected transfers

3) Security Signal in Air-Gap

  • Flags foreign content (human text, injected data)
  • Detects misuse without cloud tools

4) Auditability

  • Produces artifacts and traceable alerts
  • Supports “what happened?” with evidence

⚙️ What Makes This Strong

  • Deterministic + probabilistic detection combined
  • Fully explainable (no black box decisions)
  • Reproducible (same input → same output)
  • Portable (runs anywhere)
  • Offline-capable (no dependencies on external systems)

🧩 Clean Integration into Your Existing Document

This fits directly under your 🟪 TD Detect section.

Add this as the closing line to that section:

“TD Detect completes the platform by adding automated behavioral insight—turning raw logs into real-time visibility, anomaly detection, and actionable intelligence, even in fully air-gapped environments.”

 

Powered by Backdrop CMS
🐧 _tux