https://www.yitay.net/blog daily 1.0 2026-01-05 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/79271930-4c41-4708-905f-b4999d7cc8e4/PXL_20250117_103653504.jpg https://www.yitay.net/blog/my-year-back-at-google-deepmind monthly 0.5 2026-01-05 https://www.yitay.net/blog/returning-to-google-deepmind monthly 0.5 2024-11-26 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534472592-S0YMAAN55WVUJVCK65KR/PXL_20240616_231053661.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534472716-BZW76U5RL69TA7JB9Z1G/PXL_20240619_132750898.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534473693-55XYG3CO9DN775ZLTJGI/PXL_20240621_100555126.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534473900-WEKQXCCDCT6RCMLY9M5G/PXL_20240801_102945744.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534499855-58GLXG19DQTRR5HVDF42/PXL_20240206_041220032.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534509852-DYK9YMJNOZ8X4KU5OM3P/PXL_20230727_162510257.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534525028-WNK5LKGW7VUYQPRA3IYW/PXL_20231208_113156356.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534547203-L5GTNT6YLQ2N1FB5C6HC/PXL_20240802_123114365.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534652424-FHI7WTDB3DX1JXN3GK2T/PXL_20230728_165701158.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534656110-825T7DNWUOIWAQG6RAFV/PXL_20230727_100641354.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534690839-J2EHYI3CNPVSODLHXFX6/PXL_20240615_165437435.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732534747023-3PK5XAH0Z4EGWS9KW48Y/PXL_20230728_061523178.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532411691-HQAPUYGG7DGMMFW6ZK3F/PXL_20230923_112120389.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532411690-4JGLLUX1AZKB4TEV7N6J/PXL_20231106_121358308.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532417460-J8K172L2ZBBGYUAYHTPU/PXL_20231206_100246138.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532418471-XOSLJPBZ52MTTKUY84Q8/PXL_20231207_121112298.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532422891-EOZ4JHJ95UMMVKRL26I2/PXL_20231209_121428055.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532424371-UJF3RPTC8CI2CWX1MB5O/PXL_20240107_025146153.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532427684-UYSTOC5UVPHP4SB7QX68/PXL_20240607_094204074.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532426516-0ZBZ8N9VB43QUJ9V5774/PXL_20240818_143212511.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732532429328-8FJ5I2GKD4OWYUESKZPQ/PXL_20240821_122124526%7E2.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732548509681-BV9H0I1PAWTOB7J4456R/462569646_3933920556854774_43059256720431175_n.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732548520614-8IJ5IZT6DLAOQPBB8AVB/PXL_20240819_043527890.jpg Blog - Returning to Google DeepMind https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1732548734386-1BL6TI6PHBUJRW8PE1AV/i54d4tao.jpg Blog - Returning to Google DeepMind https://www.yitay.net/blog/model-architecture-blogpost-encoders-prefixlm-denoising monthly 0.5 2024-07-20 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/dd9d0818-bd3c-4cd7-9d12-e8570796670b/Screenshot+2024-06-26+at+5.39.11+PM.png Blog - What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives - Make it stand out Whatever it is, the way you tell your story online can make all the difference. https://www.yitay.net/blog/training-great-llms-entirely-from-ground-zero-in-the-wilderness monthly 0.5 2024-03-07 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/f20b3700-1812-42a6-9b67-bb34da564352/Screenshot+2024-03-06+at+2.35.43+AM.png Blog - Training great LLMs entirely from ground up in the wilderness as a startup - Make it stand out Training LLMs in the wilderness (image generated by Dall-E) https://www.yitay.net/blog/leaving-google-brain monthly 0.5 2023-03-30 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679845850344-C5AO4IISEJMGC6DL6FKV/PXL_20221130_192925044.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679845879219-IKGSYI1ZNMRX8Y1VZ8HR/PXL_20221204_093055377.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679845920291-6TDHNDCB2H33WEH34WPJ/PXL_20221128_164902439.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679845959192-4AFSS396UC6QI9PM4ZD3/PXL_20221128_171145560.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/b4f6f458-6102-41a7-a985-fc4efdcfef89/PXL_20221011_012748454.jpg Blog - Leaving Google Brain - Make it stand out The only one time I managed to wake up for breakfast at the Singapore office. It was delicious! Here are some other snippets of my time at the Singapore office. My desk, some food, Jason Wei's visit, TGIF. It’s a great office space truly. https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679846104377-557V0OTKF339B22CLY8A/PXL_20221211_235841159.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679846138335-WO88BS190OYDC2DDZALP/PXL_20221206_152705754.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679846188337-BO7WU8GTHFSMDHK2O79L/PXL_20221204_213016467.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679846358078-QSCOAVLTPLZ8DZCQ34GU/PXL_20221205_170446496.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679843571823-AEREG848QFZHOF6K47TD/IMG_20191105_151201.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679843601006-JI7VPUNL8AOSBWVXKCL1/IMG_20191111_083814.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679843640793-9WGLM2V2C2LZ2UEU93MK/IMG_20191212_165315_127.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679843694391-XIB64XJW48WYV9IBRQ97/IMG_20191122_084758.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679843705978-X16NZVQVL6MCZLW37G21/IMG_20191104_130844.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679843762291-E0XOK568U9RSRTCADO9G/IMG_20191230_085717.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679844790759-P3ICGK83OBSANPTLZUI4/IMG_20191105_152025.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679845171021-JNDTY3HOW5JUUVKBD1C1/IMG_20191125_080055.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679393586978-V16HAFQFYAW8VAO4ZOGA/PXL_20230317_084646790.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679393585416-3TF078UUZQ9D9HB9I458/PXL_20230317_095000586.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679393639113-O8BAND72HUG5NJN9PUZQ/PXL_20230302_072920151.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679393779221-7GMW1SI98DQIJ3W1SMNT/PXL_20220519_054329413.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679393799217-2ZP6DK8OUX01A0J9HMRT/PXL_20220826_072720505.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679393880295-II4RT397EC4W3EAY7TXT/PXL_20220913_044047177.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679406613400-4DXONTNBL0F2ZJ31L39J/PXL_20221125_043943216.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679406773676-NEK5IASB8DNEZTB7JAFU/PXL_20221122_052045494.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679406999587-0N79VTIE4HNE563KDHHC/PXL_20220708_090602110.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679407139453-90DSG27O0RDALOLP5JD6/PXL_20220923_090437981.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679407248840-RXY0PG6LEIZWXLM1OXTZ/PXL_20220826_134436525.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/1679407252290-VQJNDUC3D0REQSYHGU50/PXL_20221018_015448155.jpg Blog - Leaving Google Brain https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/c674dfb0-2390-4137-9c4c-dd70b02e234d/PXL_20220531_144430479.jpg Blog - Leaving Google Brain - Make it stand out Yes I gave a talk at AI at the loft at this beautiful place in Amsterdam! It was an amazing two weeks with Mostafa and Samira. I mentioned I started work at the MTV campus. So here are some of my noogler moments: https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/2e5afb91-2bca-433e-b4e9-e2183efbefdc/PXL_20220527_102605842.jpg Blog - Leaving Google Brain - Make it stand out Eating lunch with the Brain Amsterdam team. https://www.yitay.net/blog/flan-ul2-20b monthly 0.5 2023-03-30 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/d23d13d8-8c2e-489a-b547-ba8815c55594/Screenshot+2023-03-01+at+1.06.54+AM.png Blog - A New Open Source Flan 20B with UL2 Flan Instruction Tuning In “Scaling Instruction-Finetuned language models (Chung et al.)” (also referred to sometimes as the Flan2 paper), the key idea is to train a large language model on a collection of datasets. These datasets are phrased as instructions which enable generalization across diverse tasks. Flan has been primarily trained on academic tasks. In Flan2, we released a series of T5 models ranging from 200M to 11B parameters that have been instruction tuned with Flan. https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/7b7a8718-73cd-4142-bf7b-0175a22fcff4/Screenshot+2023-03-01+at+2.24.46+AM.png Blog - A New Open Source Flan 20B with UL2 - Flan-UL2 20B vs other Flan Models The Flan-UL2 20B checkpoint does on average +3.2% better (relatively) compared to Flan-T5 XXL on MMLU and Big-Bench Hard when compared across two different setups (direct and CoT prompting). https://www.yitay.net/blog/2022-best-nlp-papers monthly 0.5 2024-06-14 https://www.yitay.net/blog/emergence-and-scaling monthly 0.5 2023-03-30 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/711e3564-a841-4153-9ea7-8245f73d0d8d/Screen+Shot+2022-11-13+at+3.05.52+AM.png Blog - On Emergence, Scaling and Inductive Bias - Make it stand out Emergent abilities with UL2’s mixture of denoisers and Prefix language model! Inductive bias matters! https://www.yitay.net/blog/tag/scaling monthly 0.5 https://www.yitay.net/blog/tag/AGI monthly 0.5 https://www.yitay.net/blog/tag/GPUs monthly 0.5 https://www.yitay.net/blog/tag/language+model monthly 0.5 https://www.yitay.net/blog/tag/ul2 monthly 0.5 https://www.yitay.net/blog/tag/google monthly 0.5 https://www.yitay.net/blog/tag/flan-t5 monthly 0.5 https://www.yitay.net/blog/tag/flan monthly 0.5 https://www.yitay.net/blog/tag/LLM monthly 0.5 https://www.yitay.net/blog/tag/TPUs monthly 0.5 https://www.yitay.net/blog/tag/emergent+abilities monthly 0.5 https://www.yitay.net/blog/tag/Reka monthly 0.5 https://www.yitay.net/about daily 0.75 2025-11-21 https://images.squarespace-cdn.com/content/v1/636628d6a9d980577a9faf5b/79271930-4c41-4708-905f-b4999d7cc8e4/PXL_20250117_103653504.jpg https://www.yitay.net/papers daily 0.75 2025-11-15 https://www.yitay.net/thoughts daily 0.75 2025-12-31