Есть версии на 8B и 70B параметров. По сравнению с прошлой ламой тут теперь побольше словарь — 128k токенов вместо 32k (думаю отсюда и +1B параметров). А также добавили grouped query attention (GQA), чтобы это работало быстрее.
Во время обучения модели скормили 15Т токенов, это офигеть как много (по шиншилле можно и 200B версию на таком сете обучать).
Есть версии на 8B и 70B параметров. По сравнению с прошлой ламой тут теперь побольше словарь — 128k токенов вместо 32k (думаю отсюда и +1B параметров). А также добавили grouped query attention (GQA), чтобы это работало быстрее.
Во время обучения модели скормили 15Т токенов, это офигеть как много (по шиншилле можно и 200B версию на таком сете обучать).
The company maintains that it cannot act against individual or group chats, which are “private amongst their participants,” but it will respond to requests in relation to sticker sets, channels and bots which are publicly available. During the invasion of Ukraine, Pavel Durov has wrestled with this issue a lot more prominently than he has before. Channels like Donbass Insider and Bellum Acta, as reported by Foreign Policy, started pumping out pro-Russian propaganda as the invasion began. So much so that the Ukrainian National Security and Defense Council issued a statement labeling which accounts are Russian-backed. Ukrainian officials, in potential violation of the Geneva Convention, have shared imagery of dead and captured Russian soldiers on the platform. The Russian invasion of Ukraine has been a driving force in markets for the past few weeks. Investors took profits on Friday while they could ahead of the weekend, explained Tom Essaye, founder of Sevens Report Research. Saturday and Sunday could easily bring unfortunate news on the war front—and traders would rather be able to sell any recent winnings at Friday’s earlier prices than wait for a potentially lower price at Monday’s open. "There are several million Russians who can lift their head up from propaganda and try to look for other sources, and I'd say that most look for it on Telegram," he said. Update March 8, 2022: EFF has clarified that Channels and Groups are not fully encrypted, end-to-end, updated our post to link to Telegram’s FAQ for Cloud and Secret chats, updated to clarify that auto-delete is available for group and channel admins, and added some additional links.
from kr
