Créer un compteSe connecter

Recherches récentes

Aucune recherche récente

Options de recherche

Disponible uniquement lorsque vous êtes connecté.
mastouille.fr est l'un des nombreux serveurs Mastodon indépendants que vous pouvez utiliser pour participer au fédiverse.
Mastouille est une instance Mastodon durable, ouverte, et hébergée en France.

Administré par :

Statistiques du serveur :

577
comptes actifs

mastouille.fr: À proposAnnuaire des profilsPolitique de confidentialité

Mastodon: À proposTélécharger l’applicationRaccourcis clavierVoir le code sourcev4.3.4

#opticalcomputing

0 message0 participant0 message aujourd’hui
Public
Albert Cardona

"Optical transformers". Anderson et al. 2023 arxiv.org/abs/2302.10360

"we performed small-scale optical experiments with a prototype accelerator to demonstrate that Transformer operations can run on optical hardware despite noise and errors."

Claims possible energy efficiency advantage of 8,000x over conventional GPUs, with room for more.

arXiv.orgOptical TransformersThe rapidly increasing size of deep-learning models has caused renewed and growing interest in alternatives to digital computers to dramatically reduce the energy cost of running state-of-the-art neural networks. Optical matrix-vector multipliers are best suited to performing computations with very large operands, which suggests that large Transformer models could be a good target for optical computing. To test this idea, we performed small-scale optical experiments with a prototype accelerator to demonstrate that Transformer operations can run on optical hardware despite noise and errors. Using simulations, validated by our experiments, we then explored the energy efficiency of optical implementations of Transformers and identified scaling laws for model performance with respect to optical energy usage. We found that the optical energy per multiply-accumulate (MAC) scales as $\frac{1}{d}$ where $d$ is the Transformer width, an asymptotic advantage over digital systems. We conclude that with well-engineered, large-scale optical hardware, it may be possible to achieve a $100 \times$ energy-efficiency advantage for running some of the largest current Transformer models, and that if both the models and the optical hardware are scaled to the quadrillion-parameter regime, optical computers could have a $>8,000\times$ energy-efficiency advantage over state-of-the-art digital-electronic processors that achieve 300 fJ/MAC. We analyzed how these results motivate and inform the construction of future optical accelerators along with optics-amenable deep-learning approaches. With assumptions about future improvements to electronics and Transformer quantization techniques (5$\times$ cheaper memory access, double the digital--analog conversion efficiency, and 4-bit precision), we estimated that optical computers' advantage against current 300-fJ/MAC digital processors could grow to $>100,000\times$.
#transformers#OpticalComputing#MachineLearning
Public
tolortslubor

“The speed record for data transmission using a single light source and optical chip has been shattered once again. Engineers have transmitted data at a blistering rate of 1.84 petabits per second (Pbit/s), almost twice the global internet traffic per second.” newatlas.com/telecommunication #tech #technology #internet #broadband #data #bandwidth #opticalcomputing #datatransmission #opticalchip

New AtlasRecord-breaking chip can transmit entire internet's traffic per secondThe speed record for data transmission using a single light source and optical chip has been shattered once again. Engineers have transmitted data at a blistering rate of 1.84 petabits per second (Pbit/s), almost twice the global internet traffic per second.
A répondu dans un fil de discussion
Public
tolortslubor

@virginiaheffernan The trend of exponential development in info production and organization has been consistent thru five computing paradigms, and is evident in multiple digital tech streams. The log-log trend extends from the Big Bang commons.m.wikimedia.org/wiki/F #quantumcomputing #3Dtransistors #dnacomputing #memristors #EvolutionaryComputing #opticalcomputing #graphene

commons.m.wikimedia.orgFile:ParadigmShiftsFrr15Events.svg - Wikimedia Commons
Public
heise online (inoffiziell)
heise+ | Rechnen mit Licht: In Spezialbereichen tausendfach schneller als Elektronen

Lichtstrahlen zum Rechnen zu nutzen, ist nicht neu. Erstmals gibt es reelle Chancen, die altgediente Siliziumelektronik in gewissen Bereichen zu überflügeln.
Rechnen mit Licht: In Spezialbereichen tausendfach schneller als Elektronen
heise onlineRechnen mit Licht: In Spezialbereichen tausendfach schneller als ElektronenPar Thomas Brandstetter
#kunstlicheintelligenz#Prozessoren#kiprozessoren
ExplorerFlux en direct
À propos