Feed: Slashdot
Feed Link: https://slashdot.org/
---

Title: Magika 1.0 Goes Stable As Google Rebuilds Its File Detection Tool In
Rust

Link: https://developers.slashdot.org/story/25/11/0...

BrianFagioli writes: Google has released Magika 1.0, a stable version of its
AI-based file type detection tool, and rebuilt the entire engine in Rust for
speed and memory safety. The system now recognizes more than 200 file types,
up from about 100, and is better at distinguishing look-alike formats such as
JSON vs JSONL, TSV vs CSV, C vs C++, and JavaScript vs TypeScript. The team
used a 3TB training dataset and even relied on Gemini to generate synthetic
samples for rare file types, allowing Magika to handle formats that don't
have large, publicly available corpora. The tool supports Python and
TypeScript integrations and offers a native Rust command-line client. Under
the hood, Magika uses ONNX Runtime for inference and Tokio for parallel
processing, allowing it to scan around 1,000 files per second on a modern
laptop core and scale further with more CPU cores. Google says this makes
Magika suitable for security workflows, automated analysis pipelines, and
general developer tooling. Installation is a single curl or PowerShell
command, and the project remains fully open source. The project is available
on GitHub and documentation can be found here.

Read more of this story at Slashdot.

---
VRSS v2.1.180528