A few months ago, I hit a wall.
I was using a popular AI assistant to automate code snippets, summarize docs, and brainstorm blog outlines. But the more I used it, the more limitations I ran into: message caps, throttling, recurring subscription fees, and a growing unease that my conversations were being mined for metrics. Sound familiar?
So I decided to build my own AI assistant, something private, powerful, and under my full control.
Turns out, I didn’t need to reinvent the wheel. With Jan, an open-source ChatGPT-style interface that runs locally, and Pinggy, a no-fuss tunneling tool to securely share it online, I had my own AI assistant up and running in under 30 minutes.
In this post, I’ll walk you through how to set up Jan on your machine, run local AI models, connect cloud providers when needed, and share your assistant with anyone, without giving up control or privacy.
What is Jan?
At its core, Jan is a desktop AI app that feels like ChatGPT but runs entirely on your system. It doesn't need a browser, doesn’t phone home, and works even when you’re offline. You can load local models, chat with documents, hook it up to OpenAI or Claude if needed, and even expose it via an API for use in your apps or scripts.
It’s open source, fast, cleanly built, and most importantly, it’s yours.
Why Choose Jan?
Cloud AI tools are great until:
- You need to integrate them into a secure local workflow
- You're hitting token or usage limits
- You’re dealing with proprietary or confidential data
- You want to experiment with cutting-edge local models
- You just want control back
If any of that resonates, Jan is for you.
And with Pinggy, you can expose your Jan instance over the internet, perfect for testing, teamwork, or remote access without configuring DNS, ports, or reverse proxies.
Step 1: Get Jan on Your Machine
Head over to jan.ai and grab the installer for your OS (Windows, macOS, or Linux). It’s a single binary—no setup scripts, containers, or dependencies to wrestle with.
Prefer to build it yourself? Here’s how:
git clone https://github.com/menloresearch/jan
cd jan
make dev
Once launched, Jan opens a clean desktop interface. No models are bundled, so your next move is to pick one that fits your system.
Step 2: Load Your First Local Model
Click on the Hub tab to browse available models. These are in GGUF format and optimized for running locally.
Start with something lightweight like qwen2.5 if you’re on a modest setup. If you have 8GB+ RAM, try phi3.5 for better fluency and reasoning.
Models download directly inside the app and will appear in your list once ready.
Pro Tip: Begin with a smaller model to validate your setup, then scale up.
Step 3: Start Chatting (Offline)
Once a model is active, head to the New Chat tab and start typing. Whether it’s debugging help, shell scripting advice, or creative writing, Jan handles it all without a connection to the cloud.
For developers who prefer isolated tools and offline environments, this alone is a game-changer.
Step 4: Optional Cloud Integration
Jan doesn’t force you to stay offline. You can connect to external LLM providers like OpenAI, Anthropic (Claude), Google Gemini, Groq, or Cohere.
To connect:
- Open
Settings → Model Providers - Select a provider and paste your API key
- Jan will now route queries to that provider when selected
This hybrid setup lets you toggle between local and premium models depending on the task.
Step 5: Use Jan as a Local API
Need to wire Jan into a tool or internal system?
Jan includes an OpenAI-compatible REST API server. Here’s how to enable it:
- Go to
Settings → Local API Server - Add any API key (e.g.,
dev-api-key) - Click Start Server
You’ll get this confirmation:
JAN API listening at: http://127.0.0.1:1337
And now you can query it like so:
curl --location 'http://127.0.0.1:1337/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer dev-api-key' \
--data '{
"model": "qwen3:0.6b",
"messages": [{"role": "user", "content": "Explain rate limiting in APIs"}]
}'
Step 6: Share Jan Securely Using Pinggy
Want to access your Jan assistant from a remote device or share it with a teammate? Pinggy makes that dead-simple, no routers, firewalls, or cloud servers required.
With Jan running locally, open a terminal and enter:
ssh -p 443 -R0:localhost:1337 -t qr@a.pinggy.io "u:Host:localhost:1337"
Pinggy generates a public URL like:
https://x7fd12.pinggy.link
Open that in any browser, and you’re looking at your Jan instance hosted from your own machine.
Security Tips When Going Public
Jan doesn’t ship with authentication out of the box. But Pinggy lets you layer basic auth on top:
ssh -p 443 -R0:localhost:1337 -t a.pinggy.io b:username:password
Want a branded domain? Pinggy Pro lets you map to custom domains like ai.yourcompany.dev.
Real-World Use Cases
- Local DevOps Assistant: Query logs, generate scripts, and write configs securely
- Remote Coding Buddy: Share your instance with teammates for collaborative debugging
- Private Research Companion: Ask questions from sensitive PDFs or documents
- Internal Tooling: Integrate Jan’s API into dashboards, bots, or automation pipelines
Conclusion
Building your own AI assistant used to sound like a massive undertaking. But with tools like Jan and Pinggy, it’s not only possible, it’s fast, secure, and surprisingly easy.
You get total privacy, offline capability, and an interface that feels familiar. And when you need to share or scale, Pinggy extends your setup to the web without exposing your infrastructure.
For developers and IT professionals who value control, transparency, and efficiency, this might just be the setup you've been waiting for.