Самая мощная LLM в опесорсе, Mixtral 8x7B MoE от Mistral AI, теперь доступна во фреймворке Сandle* - с поддержкой квантизации. За счет квантизации модели могут работать локально на ноутбуке с 32 GB RAM.
Например, 4-битная Mixtral 8x7B MoE занимает всего 26.44GB памяти. Тогда как в bf16 на GPU модель бы заняла 112+ GB VRAM (то есть влезла бы только на H100).
*Candle - это минималистский ML-фреймворк для Rust, сфокусированный на производительности (включая поддержку GPU) и простоте использования.
- Поддерживает 2-bit, 3-bit, 4-bit, 5-bit, 6-bit and 8-bit int квантизованные модели в gguf and ggml форматах. - SIMD оптимизации для Apple Silicon и x86.
Самая мощная LLM в опесорсе, Mixtral 8x7B MoE от Mistral AI, теперь доступна во фреймворке Сandle* - с поддержкой квантизации. За счет квантизации модели могут работать локально на ноутбуке с 32 GB RAM.
Например, 4-битная Mixtral 8x7B MoE занимает всего 26.44GB памяти. Тогда как в bf16 на GPU модель бы заняла 112+ GB VRAM (то есть влезла бы только на H100).
*Candle - это минималистский ML-фреймворк для Rust, сфокусированный на производительности (включая поддержку GPU) и простоте использования.
- Поддерживает 2-bit, 3-bit, 4-bit, 5-bit, 6-bit and 8-bit int квантизованные модели в gguf and ggml форматах. - SIMD оптимизации для Apple Silicon и x86.
But because group chats and the channel features are not end-to-end encrypted, Galperin said user privacy is potentially under threat. The news also helped traders look past another report showing decades-high inflation and shake off some of the volatility from recent sessions. The Bureau of Labor Statistics' February Consumer Price Index (CPI) this week showed another surge in prices even before Russia escalated its attacks in Ukraine. The headline CPI — soaring 7.9% over last year — underscored the sticky inflationary pressures reverberating across the U.S. economy, with everything from groceries to rents and airline fares getting more expensive for everyday consumers. "And that set off kind of a battle royale for control of the platform that Durov eventually lost," said Nathalie Maréchal of the Washington advocacy group Ranking Digital Rights. DFR Lab sent the image through Microsoft Azure's Face Verification program and found that it was "highly unlikely" that the person in the second photo was the same as the first woman. The fact-checker Logically AI also found the claim to be false. The woman, Olena Kurilo, was also captured in a video after the airstrike and shown to have the injuries. The company maintains that it cannot act against individual or group chats, which are “private amongst their participants,” but it will respond to requests in relation to sticker sets, channels and bots which are publicly available. During the invasion of Ukraine, Pavel Durov has wrestled with this issue a lot more prominently than he has before. Channels like Donbass Insider and Bellum Acta, as reported by Foreign Policy, started pumping out pro-Russian propaganda as the invasion began. So much so that the Ukrainian National Security and Defense Council issued a statement labeling which accounts are Russian-backed. Ukrainian officials, in potential violation of the Geneva Convention, have shared imagery of dead and captured Russian soldiers on the platform.
from fr
