Search Results for "gpu processing"
Sort By:
Showing 58 open source projects for "gpu processing"
View related business solutions-
Run applications fast and securely in a fully managed environmentRun frontend and backend services, batch jobs, deploy websites and applications, and queue processing workloads without the need to manage infrastructure.
-
No-code automation to improve your process workflowsTransform your financial and HR operations and improve efficiency even remotely with digital, customized workflows that your team can automate and integrate with other software without the need of IT development.
-
1
GPU Puzzles
Solve puzzles. Learn CUDA
...By solving progressively more complex puzzles, learners gain a practical understanding of how parallel algorithms operate on graphics processing units. The project emphasizes experimentation and problem solving, encouraging learners to discover GPU programming techniques through trial and exploration. It can be run in cloud environments such as Google Colab, making it easy for beginners to start experimenting without configuring local GPU hardware. -
2
rembg
Rembg is a tool to remove images background
Rembg is a powerful tool that utilizes AI (specifically U^2-Net) to automatically remove backgrounds from images, offering a streamlined command-line interface and Docker support. It's ideal for batch processing and integrates smoothly into workflows -
3
FlexLLMGen
Running large language models on a single GPU
FlexLLMGen is an open-source inference engine designed to run large language models efficiently on limited hardware resources such as a single GPU. The system focuses on high-throughput generation workloads where large batches of text must be processed quickly, such as large-scale data extraction or document analysis tasks. Instead of requiring expensive multi-GPU systems, the framework uses techniques such as memory offloading, compression, and optimized batching to run large models on... -
4
Kornia
Open Source Differentiable Computer Vision Library
Kornia is a differentiable computer vision library for PyTorch. It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions. Inspired by existing packages, this library is composed by a subset of packages containing operators that can be inserted within... -
Workload Automation for Global EnterprisesRedwood lets you orchestrate securely and reliably across any application, service or server, in the cloud or on-premise, all inside a single platform.
-
5
KServe
Standardized Serverless ML Inference Platform on Kubernetes
...It aims to solve production model serving use cases by providing performant, high abstraction interfaces for common ML frameworks like Tensorflow, XGBoost, ScikitLearn, PyTorch, and ONNX. It encapsulates the complexity of autoscaling, networking, health checking, and server configuration to bring cutting edge serving features like GPU Autoscaling, Scale to Zero, and Canary Rollouts to your ML deployments. It enables a simple, pluggable, and complete story for Production ML Serving including prediction, pre-processing, post-processing and explainability. KServe is being used across various organizations. -
6
Pyreft
ReFT: Representation Finetuning for Language Models
PyreFT is a tool by Stanford NLP for fine-tuning transformer models with an emphasis on efficient, resource-conserving training and customizability for NLP tasks. -
7
NVIDIA NeMo
Toolkit for conversational AI
...NeMo uses PyTorch Lightning for easy and performant multi-GPU/multi-node mixed-precision training. Supported models: Jasper, QuartzNet, CitriNet, Conformer-CTC, Conformer-Transducer, Squeezeformer-CTC, Squeezeformer-Transducer, ContextNet, LSTM-Transducer (RNNT), LSTM-CTC. NGC collection of pre-trained speech processing models. -
8
CUDA Python
Performance meets Productivity
...It integrates tightly with the broader Python GPU ecosystem, including Numba for kernel compilation and CCCL for parallel primitives, allowing developers to write performant code without leaving Python. The toolkit also includes utilities for profiling, memory management, distributed computing, and numerical operations, making it suitable for scientific computing, AI, and data processing workloads. -
9
HunyuanVideo
HunyuanVideo: A Systematic Framework For Large Video Generation Model
HunyuanVideo is a cutting-edge framework designed for large-scale video generation, leveraging advanced AI techniques to synthesize videos from various inputs. It is implemented in PyTorch, providing pre-trained model weights and inference code for efficient deployment. The framework aims to push the boundaries of video generation quality, incorporating multiple innovative approaches to improve the realism and coherence of the generated content. Release of FP8 model weights to reduce GPU... -
Your go-to FinOps platformUnlike reporting-only FinOps tools, FinOpsly unifies cloud (AWS, Azure, GCP), data (Snowflake, Databricks, BigQuery), and AI costs into a single system of action — enabling teams to plan spend before it happens, automate optimization safely, and prove value in weeks, not quarters.
-
10
clip-retrieval
Easily compute clip embeddings and build a clip retrieval system
...It allows developers to compute embeddings for both images and text efficiently and then index them for fast similarity search across massive datasets. The system is optimized for performance and scalability, capable of processing tens or even hundreds of millions of embeddings using GPU acceleration. It includes components for inference, indexing, filtering, and serving results through APIs, making it a complete pipeline for building production-ready retrieval systems. The framework also supports querying by image, text, or embedding, enabling flexible use cases such as reverse image search or multimodal content discovery. ... -
11
Colossal-AI
Making large AI models cheaper, faster and more accessible
The Transformer architecture has improved the performance of deep learning models in domains such as Computer Vision and Natural Language Processing. Together with better performance come larger model sizes. This imposes challenges to the memory wall of the current accelerator hardware such as GPU. It is never ideal to train large models such as Vision Transformer, BERT, and GPT on a single GPU or a single machine. There is an urgent demand to train models in a distributed environment. ... -
12
Text Generation Inference
Large Language Model Text Generation Inference
Text Generation Inference is a high-performance inference server for text generation models, optimized for Hugging Face's Transformers. It is designed to serve large language models efficiently with optimizations for performance and scalability. -
13
clone-voice
A sound cloning tool with a web interface, using your voice
Clone-voice is a local voice-cloning tool that lets you synthesize speech in any target voice or convert one recording into another voice using the same timbre. It is built around Coqui’s XTTS-v2 model, so it inherits multilingual support and modern neural TTS quality while wrapping it in a user-friendly desktop workflow. The app is designed to be very easy to use: you download a precompiled package, double-click app.exe, and it launches a browser-based web interface where you control... -
14
Insanely Fast Whisper
An opinionated CLI to transcribe Audio files w/ Whisper on-device
Insanely Fast Whisper is a high-performance command-line tool designed to dramatically accelerate speech-to-text transcription using OpenAI’s Whisper models on local hardware. It leverages modern optimizations such as batch processing, mixed precision, and advanced attention mechanisms like Flash Attention to significantly reduce inference time while maintaining high transcription accuracy. The project is built on top of the Transformers ecosystem and integrates with libraries such as Optimum to maximize GPU efficiency. It is specifically engineered for environments with CUDA-enabled GPUs or Apple Silicon devices, allowing users to process hours of audio in minutes or even seconds depending on hardware capabilities. ... -
15
stt
Voice Recognition to Text Tool
...The project is designed to be easy to deploy: you can run a local Python server that exposes an HTTP API for uploading audio/video files and retrieving transcriptions in different formats. It supports GPU acceleration if available, enabling faster processing on compatible hardware but still offers reliable performance on CPUs alone. -
16
Instill Core
Instill Core is a full-stack AI infrastructure tool for data
Instill Core is an open-source, full-stack AI infrastructure platform designed to orchestrate data pipelines, machine learning models, and unstructured data processing into a unified, production-ready system. It provides an end-to-end solution that enables developers to build, deploy, and manage AI-powered applications without needing to manually stitch together multiple tools across the data and model lifecycle. The platform focuses heavily on handling unstructured data such as documents, images, audio, and video, transforming them into AI-ready formats through integrated ETL pipelines and processing workflows. ... -
17
API-for-Open-LLM
Openai style api for open large language models
API-for-Open-LLM is a lightweight API server designed for deploying and serving open large language models (LLMs), offering a simple way to integrate LLMs into applications. -
18
TorchAudio
Data manipulation and transformation for audio signal processing
The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the same philosophy of providing strong GPU acceleration, having a focus on trainable features through the autograd system, and having consistent style (tensor names and dimension names). Therefore, it is primarily a machine learning library and not a general signal processing library. The benefits of PyTorch can be seen in torchaudio through having all the computations be through PyTorch operations which makes it easy to use and feel like a natural extension. -
19
Depth Pro
Sharp Monocular Metric Depth in Less Than a Second
Depth Pro is a foundation model for zero-shot metric monocular depth estimation, producing sharp, high-frequency depth maps with absolute scale from a single image. Unlike many prior approaches, it does not require camera intrinsics or extra metadata, yet still outputs metric depth suitable for downstream 3D tasks. Apple highlights both accuracy and speed: the model can synthesize a ~2.25-megapixel depth map in around 0.3 seconds on a standard GPU, enabling near real-time applications. The... -
20
LiveAvatar
Streaming Real-time Audio-Driven Avatar Generation
...It implements techniques from state-of-the-art diffusion-based avatar modeling to support infinite-length continuous video generation with low latency, enabling interactive AI avatars that maintain continuity and realism over extended sessions. The project co-designs algorithms and system optimizations, such as block-wise autoregressive processing and fast sampling strategies, to deliver real-time frame rates (e.g., ~45 FPS on appropriate GPU clusters) while handling non-stop generation without quality degradation. LiveAvatar focuses on delivering not just high-quality visuals but also the responsiveness necessary for immersive conversational experiences, making it suitable for advanced AI agents, virtual assistants, and interactive streaming contexts. -
21
TensorRT Node for ComfyUI
Enables the best performance on NVIDIA RTX Graphics Cards
...It bridges the gap between ComfyUI’s flexible, node-based workflows and TensorRT’s highly optimized engine format. The result is that complex diffusion or image-processing graphs can be accelerated without the user having to rewrite the pipeline. The repo typically includes instructions for converting models to TensorRT engines and for wiring those engines into ComfyUI nodes. This is particularly attractive for power users who run many generations or who host ComfyUI on dedicated hardware and want to squeeze out every bit of GPU performance. ... -
22
Faster Whisper
Faster Whisper transcription with CTranslate2
Faster Whisper is an optimized implementation of the Whisper speech recognition model designed to deliver significantly faster inference while maintaining comparable accuracy. It leverages efficient inference engines and optimized computation strategies to reduce latency and resource consumption. The system is particularly useful for real-time or large-scale transcription tasks where performance is critical. It supports multiple model sizes, allowing users to balance speed and accuracy based... -
23
LitServe
Minimal Python framework for scalable AI inference servers fast
LitServe is a minimal Python framework designed for building custom AI inference servers with full control over how models are executed and served. It allows developers to define their own inference logic, making it suitable for complex systems such as multi-model pipelines, agents, and retrieval-augmented generation workflows. Unlike traditional serving tools that enforce rigid abstractions, LitServe focuses on flexibility by letting users control request handling, batching strategies, and... -
24
NVIDIA Generative AI Examples
Generative AI reference workflows
...The repository includes examples covering topics such as retrieval-augmented generation pipelines, agent-based workflows, and multimodal AI applications that combine text, vision, and data processing. Many of the examples show how to deploy AI services using containerized environments, GPU acceleration, and microservices that can scale across modern infrastructure. Developers can explore sample chatbot applications, document question-answering systems, and knowledge-base pipelines that illustrate how generative AI can interact with external data sources. -
25
BentoML
Unified Model Serving Framework
...Orchestrate distributed inference graph with multiple models via Yatai on Kubernetes. Easily configure CUDA dependencies for running inference with GPU. Automatically generate docker images for production deployment.
