AI Sight Quest is an iOS application powered by Artificial Intelligence that leverages Machine Learning models and Apple's Frameworks. It enables users to take a photo or select them from their gallery within the app and capture the text using Apple's Vision Framework and locate relevant passages in a document by asking the Bidirectional Encoder Representations from Transformers (BERT) model a question.
Discover AI Sight Quest
All computations and processing are performed locally on the device, ensuring user data privacy and security. There's no reliance on external servers or network API calls. Every calculation and processing task is handled by the device itself, enhancing both performance and privacy.
The app generates answers directly within scanned documents using the Core ML Framework and a question-answering AI model.
Interacting with this captured text is easy; users can edit or even play it aloud, facilitated by text playback using Apple's Speech Framework.
Users can customize their reading experience. They can change the font size or switch between different dark or light color schemes, ensuring optimal readability in various environments.
Also, to achieve our versatile app design, we opted for vector-based animations over traditional methods, prioritizing performance. Leveraging Lottie animations ensured seamless visuals while maintaining high efficiency.
By integrating Apple's TipKit Framework into the application, users benefit from valuable tips and insights, enhancing their experience and effectively guiding them through the app's features.
To set up and run the application, please follow these steps:
- Begin by cloning the repository onto your local machine.
- Owing to the substantial size of the BERT model, it is not directly integrated into the project. You may acquire it by downloading it from the following link:
-
Rename the downloaded BERT model to
BERTSQUAD.mlmodel -
Open the project in Xcode and integrate the downloaded model into the project structure at the designated location:
./AISightQuest/Resources/BERT Core ML Model Resources -
Finally, build and launch the application either on the iOS simulator or a connected device.
Feel free to share your ideas or any other problems. Pull requests are welcome! For major changes, please open an issue first to discuss what you would like to change.
AI Sight Quest is released under the MIT license. See LICENSE for more information.