Add initial work from Codex
This commit is contained in:
144
README.md
144
README.md
@@ -1,2 +1,144 @@
|
||||
# zavrsni-rad-otel-app
|
||||
# OTel BI Forecast App
|
||||
|
||||
OpenTelemetry-instrumented BI platform with microservices, frontend OIDC login plus backend token validation, read-only MSSQL data warehouse access, and PostgreSQL persistence for writable app data.
|
||||
|
||||
## Architecture
|
||||
|
||||
- Frontend: React + TypeScript (`frontend/`)
|
||||
- Backend microservices (`backend/microservices/`):
|
||||
- `api_gateway`: public API, frontend JWT validation, internal token minting, routing/audit forwarding
|
||||
- `bi_query`: read-only MSSQL warehouse queries
|
||||
- `analytics`: forecasting, rankings, recommendations
|
||||
- `persistence`: PostgreSQL writes/reads for app data
|
||||
- Data sources:
|
||||
- MSSQL (`WorldWideImporters`, `AdventureWorks2022DWH`) read-only only
|
||||
- PostgreSQL writable app store (`audit_logs`, `forecast_runs`, `ranking_runs`, `recommendation_runs`)
|
||||
- Observability: OTLP/HTTP to Grafana Alloy (`/v1/traces`, `/v1/metrics`)
|
||||
|
||||
## Authentication Model
|
||||
|
||||
- Frontend uses OIDC Authorization Code + PKCE.
|
||||
- `api_gateway` validates frontend bearer JWT (`iss`, `aud`, signature, expiry, optional scopes) against configured JWKS.
|
||||
- `api_gateway` mints short-lived internal service tokens (`x-internal-service-token`) for service-to-service calls.
|
||||
- Internal services (`analytics`, `bi_query`, `persistence`) require valid internal token on non-health endpoints and enforce issuer/type checks.
|
||||
- Combine with K8s network controls (ClusterIP, NetworkPolicy, mTLS/service mesh where available).
|
||||
|
||||
Frontend uses OIDC Authorization Code + PKCE with:
|
||||
- `VITE_OIDC_ENABLED=true`
|
||||
- `VITE_OIDC_AUTHORITY=<issuer-base-url>`
|
||||
- `VITE_OIDC_CLIENT_ID=<frontend-client-id>`
|
||||
- `VITE_OIDC_REDIRECT_URI=<frontend-url>`
|
||||
- `VITE_OIDC_POST_LOGOUT_REDIRECT_URI=<frontend-url>`
|
||||
- `VITE_OIDC_SCOPE=openid profile email`
|
||||
|
||||
Backend security env:
|
||||
- `REQUIRE_FRONTEND_AUTH=true`
|
||||
- `FRONTEND_JWT_ISSUER_URL=<oidc-issuer>`
|
||||
- `FRONTEND_JWT_JWKS_URL=<issuer-jwks-url>`
|
||||
- `FRONTEND_JWT_AUDIENCE=<api-audience>`
|
||||
- `FRONTEND_REQUIRED_SCOPES=<space-separated>`
|
||||
- `INTERNAL_SERVICE_SHARED_SECRET=<strong-random-secret-at-least-32-bytes>`
|
||||
- `INTERNAL_SERVICE_ALLOWED_ISSUERS=api-gateway`
|
||||
- `MSSQL_TRUST_SERVER_CERTIFICATE=false` and `POSTGRES_SSLMODE=require` for production TLS validation
|
||||
|
||||
## Local Run (Microservices)
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e .
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Run services in separate terminals:
|
||||
|
||||
```bash
|
||||
uvicorn microservices.persistence.main:app --host 0.0.0.0 --port 8103 --reload
|
||||
uvicorn microservices.bi_query.main:app --host 0.0.0.0 --port 8101 --reload
|
||||
uvicorn microservices.analytics.main:app --host 0.0.0.0 --port 8102 --reload
|
||||
uvicorn microservices.api_gateway.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
```
|
||||
|
||||
Frontend:
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
cp .env.example .env
|
||||
npm run dev
|
||||
```
|
||||
|
||||
Set:
|
||||
- `VITE_API_BASE_URL=http://localhost:8000`
|
||||
- `VITE_OTEL_COLLECTOR_ENDPOINT=http://alloy.monitoring.svc.cluster.local:4318`
|
||||
|
||||
Frontend sends `Authorization: Bearer <token>` from the active OIDC session.
|
||||
|
||||
## API Endpoints (via Gateway)
|
||||
|
||||
- `GET /api/health`
|
||||
- `GET /api/telemetry/status`
|
||||
- `GET /api/kpis`
|
||||
- `GET /api/history?days_back=365`
|
||||
- `GET /api/forecasts?days=30`
|
||||
- `GET /api/rankings?top_n=10`
|
||||
- `GET /api/recommendations`
|
||||
- `GET /api/dashboard`
|
||||
- `GET /api/storage/audit-logs?limit=50`
|
||||
- `GET /api/storage/forecasts?limit=50`
|
||||
- `GET /api/storage/rankings?limit=50`
|
||||
- `GET /api/storage/recommendations?limit=50`
|
||||
|
||||
## K8s Deployment
|
||||
|
||||
Example manifest:
|
||||
- `k8s/microservices.yaml`
|
||||
|
||||
It includes:
|
||||
- namespace, config map, secret
|
||||
- deployments/services for `api-gateway`, `bi-query`, `analytics`, `persistence`
|
||||
- Alloy endpoint wiring via `OTEL_COLLECTOR_ENDPOINT`
|
||||
- frontend JWT validation config and internal token secret wiring
|
||||
- hardened pod security defaults (`runAsNonRoot`, dropped capabilities, `seccompProfile: RuntimeDefault`, no auto-mounted service account token)
|
||||
|
||||
## Read-Only Guarantee
|
||||
|
||||
- MSSQL connections enforce `ApplicationIntent=ReadOnly`.
|
||||
- Warehouse query layer only accepts `SELECT`/`WITH`.
|
||||
- Writable operations are isolated to PostgreSQL only.
|
||||
- Use SQL Server account with `SELECT` grants only.
|
||||
|
||||
## OTel Coverage
|
||||
|
||||
- Frontend:
|
||||
- W3C trace/baggage propagation
|
||||
- document-load, user-interaction, fetch, XHR instrumentation
|
||||
- manual dashboard spans
|
||||
- Backend services:
|
||||
- FastAPI request spans
|
||||
- HTTP client spans for service-to-service calls
|
||||
- SQLAlchemy spans (MSSQL and PostgreSQL)
|
||||
- manual analytics + persistence spans
|
||||
- audit/snapshot persistence telemetry
|
||||
|
||||
## Verification
|
||||
|
||||
1. Call `GET /api/telemetry/status` with a valid frontend bearer token.
|
||||
2. Confirm response has non-null `trace_id` and `span_id`.
|
||||
3. Trigger `GET /api/dashboard`; then verify records in `GET /api/storage/audit-logs`.
|
||||
4. In Grafana/Tempo, confirm trace path includes:
|
||||
- `api-gateway` span
|
||||
- `analytics` span
|
||||
- `bi-query` MSSQL spans
|
||||
- `persistence` PostgreSQL spans
|
||||
5. Call internal service endpoint directly without `x-internal-service-token` and verify it returns `401`.
|
||||
|
||||
## Optional Tests
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
source .venv/bin/activate
|
||||
pip install -e .[dev]
|
||||
pytest
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user