Немного странный анонс моделей с приговоркой "статья будет чуть позже", meta раньше так не делали, но всё равно модели очень крутые и уже доступны
Детали: 1. 16K GPU 🤯 1. 15T токенов 🤯🤯 1. Веса моделей на 8B и 70B параметров уже доступны 🎉 1. Тренируют модель на 405B параметров (без MoE) 🤯 1. 8K длина контекста
1. Архиткетурно самые большие отличия: Grouped Query Attention и 128K vocab size 1. Для тренировки оценивали scaling laws на разных доменах датасета (и на downstream задачах) после чего из них высчитывали оптимальное взвешивание
Бенчмарки: 1. На MMLU, Llama 3 8B работает на уровне PALM-540B и Chinchilla 70B 1. Там же Llama 70B обходит Claude 3 Sonnet и Mistral Large
Немного странный анонс моделей с приговоркой "статья будет чуть позже", meta раньше так не делали, но всё равно модели очень крутые и уже доступны
Детали: 1. 16K GPU 🤯 1. 15T токенов 🤯🤯 1. Веса моделей на 8B и 70B параметров уже доступны 🎉 1. Тренируют модель на 405B параметров (без MoE) 🤯 1. 8K длина контекста
1. Архиткетурно самые большие отличия: Grouped Query Attention и 128K vocab size 1. Для тренировки оценивали scaling laws на разных доменах датасета (и на downstream задачах) после чего из них высчитывали оптимальное взвешивание
Бенчмарки: 1. На MMLU, Llama 3 8B работает на уровне PALM-540B и Chinchilla 70B 1. Там же Llama 70B обходит Claude 3 Sonnet и Mistral Large
Telegram has become more interventionist over time, and has steadily increased its efforts to shut down these accounts. But this has also meant that the company has also engaged with lawmakers more generally, although it maintains that it doesn’t do so willingly. For instance, in September 2021, Telegram reportedly blocked a chat bot in support of (Putin critic) Alexei Navalny during Russia’s most recent parliamentary elections. Pavel Durov was quoted at the time saying that the company was obliged to follow a “legitimate” law of the land. He added that as Apple and Google both follow the law, to violate it would give both platforms a reason to boot the messenger from its stores. In the United States, Telegram's lower public profile has helped it mostly avoid high level scrutiny from Congress, but it has not gone unnoticed. Pavel Durov, Telegram's CEO, is known as "the Russian Mark Zuckerberg," for co-founding VKontakte, which is Russian for "in touch," a Facebook imitator that became the country's most popular social networking site. "There are a lot of things that Telegram could have been doing this whole time. And they know exactly what they are and they've chosen not to do them. That's why I don't trust them," she said. Telegram users are able to send files of any type up to 2GB each and access them from any device, with no limit on cloud storage, which has made downloading files more popular on the platform.
from hk
