TinyChatEngine
TinyChatEngine: On-Device LLM Inference Library
About
Running large language models (LLMs) and visual language models (VLMs) on the edge is useful: copilot services (coding, office, smart reply) on laptops, cars, robots, and more. Users can get instant responses with better privacy, as the data is local.
Open Source Health
- Stars
- 962
- Forks
- 102
- License
- MIT
- Last commit
- 2 years ago
Resources & Links
Related Categories
Vendor
MIT HAN Lab
Efficient AI Computing. PI: Song Han
Quick Links
Open Source
More by MIT HAN Lab
Related Products
Rclip
Semantic photo search for the command line
Top category match
ChineseNER
A neural network model for Chinese named entity recognition
Top category match
Head Pose Estimation
Realtime human head pose estimation with ONNXRuntime and OpenCV.
Top category match
Video Subtitle Extractor
视频硬字幕提取，生成srt文件。无需申请第三方API，本地实现文本识别。基于深度学习的视频字幕提取框架，包含字幕区域检测、字幕内容提取。A GUI tool for extracting hard-coded subtitle (hardsub) from videos and generating srt files.
Top category match
U 2 Net
The code for our newly accepted paper in Pattern Recognition 2020: "U^2-Net: Going Deeper with Nested U-Structure for Salient Object Detection."
Top category match