WondTech brings you news about a clever way to securely track how your AI models are performing. This new pipeline lets teams view AI evaluation data without ever exposing their sensitive internal systems to the internet.
Have you ever wondered how to keep an eye on your AI models’ performance without accidentally putting your valuable internal data at risk? Well, we’ve got some exciting news for you! A new approach has been documented for building a secure, self-hosted pipeline specifically designed to watch over AI evaluation platforms.
This isn't about fancy new dashboards or complex scoring models just yet. Instead, the brilliant minds behind this focused on a more fundamental question: How can a team safely see what's happening with their AI evaluations – the 'telemetry' data – without letting the outside world peek at their virtual machines, databases, passwords, or other internal services? Their answer is a smart, secure, and self-hosted path for 'observability,' which basically means giving you visibility into your systems.
What does this mean for you? Imagine being able to check on your AI models' progress through a simple web browser interface. This new pipeline makes it possible, all while keeping your actual infrastructure completely private. It uses a clever setup: when a team member wants to view data, they go through a protected entry point. This entry point verifies their identity using a one-time passcode sent to their company email, all managed securely at the 'edge' of the network.
From there, a Cloudflare Tunnel creates a secure, outbound-only connection from a private Azure virtual machine to that public edge. Inside your Azure VM, a Caddy reverse proxy routes traffic locally. Your AI evaluation platform then sends out 'observability spans' – safe bits of operational data – which are then processed by an OpenTelemetry Collector. Crucially, this collector applies redaction rules, meaning it automatically strips out any sensitive information before the data even reaches the visualization tool, Arize Phoenix.
So, you get a clean, safe browser view of all your trace and evaluation metadata. The core idea is simple: give teams the insights they need to understand their AI, but keep the underlying technical details and sensitive infrastructure hidden away. It’s important to note that this is currently an observability and access pipeline; it’s not yet a complete system for running evaluation jobs or onboarding full projects. But it’s a massive step towards secure and private AI development!
This isn't about fancy new dashboards or complex scoring models just yet. Instead, the brilliant minds behind this focused on a more fundamental question: How can a team safely see what's happening with their AI evaluations – the 'telemetry' data – without letting the outside world peek at their virtual machines, databases, passwords, or other internal services? Their answer is a smart, secure, and self-hosted path for 'observability,' which basically means giving you visibility into your systems.
What does this mean for you? Imagine being able to check on your AI models' progress through a simple web browser interface. This new pipeline makes it possible, all while keeping your actual infrastructure completely private. It uses a clever setup: when a team member wants to view data, they go through a protected entry point. This entry point verifies their identity using a one-time passcode sent to their company email, all managed securely at the 'edge' of the network.
From there, a Cloudflare Tunnel creates a secure, outbound-only connection from a private Azure virtual machine to that public edge. Inside your Azure VM, a Caddy reverse proxy routes traffic locally. Your AI evaluation platform then sends out 'observability spans' – safe bits of operational data – which are then processed by an OpenTelemetry Collector. Crucially, this collector applies redaction rules, meaning it automatically strips out any sensitive information before the data even reaches the visualization tool, Arize Phoenix.
So, you get a clean, safe browser view of all your trace and evaluation metadata. The core idea is simple: give teams the insights they need to understand their AI, but keep the underlying technical details and sensitive infrastructure hidden away. It’s important to note that this is currently an observability and access pipeline; it’s not yet a complete system for running evaluation jobs or onboarding full projects. But it’s a massive step towards secure and private AI development!
