Немного странный анонс моделей с приговоркой "статья будет чуть позже", meta раньше так не делали, но всё равно модели очень крутые и уже доступны
Детали: 1. 16K GPU 🤯 1. 15T токенов 🤯🤯 1. Веса моделей на 8B и 70B параметров уже доступны 🎉 1. Тренируют модель на 405B параметров (без MoE) 🤯 1. 8K длина контекста
1. Архиткетурно самые большие отличия: Grouped Query Attention и 128K vocab size 1. Для тренировки оценивали scaling laws на разных доменах датасета (и на downstream задачах) после чего из них высчитывали оптимальное взвешивание
Бенчмарки: 1. На MMLU, Llama 3 8B работает на уровне PALM-540B и Chinchilla 70B 1. Там же Llama 70B обходит Claude 3 Sonnet и Mistral Large
Немного странный анонс моделей с приговоркой "статья будет чуть позже", meta раньше так не делали, но всё равно модели очень крутые и уже доступны
Детали: 1. 16K GPU 🤯 1. 15T токенов 🤯🤯 1. Веса моделей на 8B и 70B параметров уже доступны 🎉 1. Тренируют модель на 405B параметров (без MoE) 🤯 1. 8K длина контекста
1. Архиткетурно самые большие отличия: Grouped Query Attention и 128K vocab size 1. Для тренировки оценивали scaling laws на разных доменах датасета (и на downstream задачах) после чего из них высчитывали оптимальное взвешивание
Бенчмарки: 1. На MMLU, Llama 3 8B работает на уровне PALM-540B и Chinchilla 70B 1. Там же Llama 70B обходит Claude 3 Sonnet и Mistral Large
As a result, the pandemic saw many newcomers to Telegram, including prominent anti-vaccine activists who used the app's hands-off approach to share false information on shots, a study from the Institute for Strategic Dialogue shows. These entities are reportedly operating nine Telegram channels with more than five million subscribers to whom they were making recommendations on selected listed scrips. Such recommendations induced the investors to deal in the said scrips, thereby creating artificial volume and price rise. The next bit isn’t clear, but Durov reportedly claimed that his resignation, dated March 21st, was an April Fools’ prank. TechCrunch implies that it was a matter of principle, but it’s hard to be clear on the wheres, whos and whys. Similarly, on April 17th, the Moscow Times quoted Durov as saying that he quit the company after being pressured to reveal account details about Ukrainians protesting the then-president Viktor Yanukovych. But Telegram says people want to keep their chat history when they get a new phone, and they like having a data backup that will sync their chats across multiple devices. And that is why they let people choose whether they want their messages to be encrypted or not. When not turned on, though, chats are stored on Telegram's services, which are scattered throughout the world. But it has "disclosed 0 bytes of user data to third parties, including governments," Telegram states on its website. Telegram has become more interventionist over time, and has steadily increased its efforts to shut down these accounts. But this has also meant that the company has also engaged with lawmakers more generally, although it maintains that it doesn’t do so willingly. For instance, in September 2021, Telegram reportedly blocked a chat bot in support of (Putin critic) Alexei Navalny during Russia’s most recent parliamentary elections. Pavel Durov was quoted at the time saying that the company was obliged to follow a “legitimate” law of the land. He added that as Apple and Google both follow the law, to violate it would give both platforms a reason to boot the messenger from its stores.
from cn
