The Architecture I Built vs. The Architecture I Could Deploy
I participated in a competition and it gave me an insight into something interesting and it led me to build Vicquant. An offline financial advsior app but when i was building it, i ran into a problem and and this aid me in solving real world problem and be innovative for the time being.
The article discusses the development of Vicquant, an offline financial advisor app, and the challenges faced in deploying a dual-model architecture with zero cloud dependency.
What happened
The author developed Vicquant, an offline financial advisor app, using a dual-model architecture with Llama-3.1-8B-Instruct and Phi-3.5-mini. However, due to hosting constraints, the app currently relies on OpenRouter for AI responses, with offline mode disabled. The author is transparent about this limitation to maintain credibility.
Why it matters
This article matters to AI developers and software engineers because it highlights the gap between designing an ideal architecture and deploying it in practice. It also demonstrates how to balance performance and practical constraints in real-world applications.
- Designing an ideal architecture is different from deploying it in practice.
- Transparency about limitations is crucial for maintaining credibility.
- Offline deployment can be achieved with local models, but hosting constraints may require cloud-based solutions.
- Deterministic logic can be used for core functionalities, making them independent of the AI model used.
- Model routing can help balance performance and resource constraints.
The bigger picture
The article shows how developers can balance innovation with practical deployment constraints. It also emphasizes the importance of transparency in technical communication to maintain credibility.
The article concludes that while the ideal architecture for Vicquant is offline and uses local models, deployment constraints require cloud-based solutions. The author is transparent about this limitation and plans to return to offline deployment once hosting constraints are resolved.
Technical details
- Dual local-model setup with Llama-3.1-8B-Instruct and Phi-3.5-mini.
- Offline deployment using llama.cpp on the user's device.
- Model routing logic for switching between models based on performance needs.
- Integration with OpenRouter for cloud-based AI responses.
- Deterministic Python-based signal generation and backtesting.
FAQ
Why is Vicquant using a dual-model setup?
The dual-model setup allows for balancing reasoning quality and response speed, as one model cannot optimize for both.
Why is the app currently using OpenRouter?
Due to hosting constraints, the app currently relies on OpenRouter for AI responses, with offline mode disabled in the live deployment.
What is the impact of using a local model versus a cloud-based model?
Using a local model allows for offline deployment, but hosting constraints may require cloud-based solutions for practical deployment.
How does the app ensure the accuracy of its financial signals?
The app's core logic, including signal generation and backtesting, is deterministic and unaffected by the model used.
When will offline mode be available again?
Offline mode will return once the hosting constraint is resolved, and the author will post an update when it happens.
This summary and analysis were generated with AI from the original article and may contain errors. Read the original for the full story.
Read the full story
Originally published by Dev.to AI on Oct 4, 2026.
Comments
Sign in to join the discussion.