InfoQ

Google Releases LMEval, an Open-Source Cross-Provider LLM Evaluation Tool

For cross-provider support, it is critical that evaluation benchmarks can be defined once and reused across multiple models, despite differences in their APIs. To this end, LMEval uses LiteLLM, a ...
Forbes

The Open-Source LLM Revolution: Transforming Enterprise AI For A New Era

On January 20, 2025, Chinese AI startup DeepSeek unveiled R1, an open-source large language model (LLM) that is redefining industry expectations. Designed to offer performance on par with proprietary ...
InfoWorld

AiSuite: An open-source AI gateway for unified LLM access

Python project from Andrew Ng provides a streamlined approach to working with multiple LLM providers, addressing a significant pain point in the AI development workflow. The proliferation of large ...
InfoQ

Anthropic Publishes Model Context Protocol Specification for LLM App Integration

Anthropic recently released their Model Context Protocol (MCP), an open standard describing a protocol for integrating external resources and tools with LLM apps. The release includes SDKs ...
InfoWorld

MarkItDown: Microsoft’s open-source tool for Markdown conversion

The rapid evolution of generative AI has created a pressing need for tools that can efficiently prepare diverse data sources for large language models (LLMs). Transforming information that is encoded ...
Forbes

LLM-As-A-Judge: What To Expect From Using AI To Evaluate AI

Jeffrey Ip is a former engineer who loves solving complex problems. He also cofounded Confident AI, a YC-backed startup. Every day, enterprise AI systems generate millions of responses that no human ...
Diginomica

Want better LLM results? Then it's time for AI evaluation tools - learning from Galileo's RAG and agent metrics

A consistent media flood of sensational hallucinations from the big AI chatbots. Widespread fear of job loss, especially due to lack of proper communication from leadership - and relentless overhyping ...