Éî¶Èѧϰ֮¸¸Hinton±¸ÊÜÖõÄ¿µÄCapsuleÂÛÎĽñÕýʽ¹«²¼
2017-10-28 10:18:44   À´Ô´£º37¶ÈҽѧÍø   ×÷Õߣº  ÆÀÂÛ£º0 µã»÷£º

ÔÚÈ˹¤ÖÇÄÜѧ½ç£¬Geoffrey HintonÓµÓзdz£³ç¸ßµÄµØ룬ÉõÖÁ±»ÓþΪ¸ÃÁìÓòµÄ°®Òò˹̹¡£×÷Ϊ“Éî¶Èѧϰ”Ö®¸¸£¬Ò²ÕýÊÇÕâ¸ö¼¼ÊõÈÃÈ˹¤ÖÇÄÜ·¢Õ¹µ½½ñÌìÕâ°ãÖËÊÖ¿ÉÈÈ¡£ÔÚÈ˹¤ÖÇÄÜÁìÓò×¼âµÄÑо¿ÈËÔ±µ±ÖУ¬HintonµÄÒýÓÃÂÊ×î¸ß£¬³¬¹ýÁËÅÅÔÚËûºóÃæÈýλÑо¿ÈËÔ±µÄ×ܺ͡£Ä¿Ç°£¬ËûµÄѧÉúºÍ²©Ê¿ºóÁìµ¼×ÅÆ»¹û¡¢FacebookÒÔ¼°OpenAIµÄÈ˹¤ÖÇÄÜʵÑéÊÒ£¬¶øHinton±¾ÈËÊǹȸè´óÄÔ£¨Google Brain£©È˹¤ÖÇÄÜÍŶӵÄÊ×ϯ¿Æѧ¼Ò¡£

ÊÂʵÉÏ£¬È˹¤ÖÇÄÜÔÚ×î½üÊ®ÄêÀïÈ¡µÃµÄ¼¸ºõÿһ¸ö³É¾Í£¬°üÀ¨ÓïÒôʶ±ð¡¢Í¼Ïñʶ±ð£¬ÒÔ¼°²©ÞÄ£¬ÔÚijÖ̶ֳÈÉ϶¼ÄÜ×·Ëݵ½ Hinton µÄ¹¤×÷¡£

1986Ä꣬HiltonÁªºÏͬÊ´óÎÀ·Â³Ä·¹þÌØ£¨David Rumelhart£©ºÍÂÞÄɵ·ÍþÁ®Ä·Ë¹£¨Ronald Williams£©£¬·¢±íÁËһƪͻÆÆÐÔµÄÂÛÎÄ£¬Ïêϸ½éÉÜÁËÒ»ÖÖ½Ð×÷“·´Ïò´«²¥”£¨backpropagation£¬¼ò³Æbackprop£©µÄ¼¼Êõ¡£ÆÕÁÖ˹¶Ù¼ÆËãÐÄÀíѧ¼ÒÇǶ÷·¿Æ¶÷£¨Jon Cohen£©½«·´Ïò´«²¥¶¨ÒåΪ“ËùÓÐÉî¶Èѧϰ¼¼ÊõµÄ»ù´¡”¡£¹é¸ù½áµ×£¬½ñÌìµÄÈ˹¤ÖÇÄܾÍÊÇÉî¶Èѧϰ£¬¶øÉî¶Èѧϰ¾ÍÊÇ·´Ïò´«²¥£¬ËäÈ»ÎÒÃǺÜÄÑÏàÐÅ·´Ïò´«²¥ÒѾ­³öÏÖÁË30¶àÄê¡£

È»¶ø£¬¾ÍÔÚÉϸöÔ£¬HintonÔÚ¶àÂ׶àµÄÒ»³¡ AI »áÒéÉÏ£¬Ãæ¶ÔýÌå Axios µÄ²É·Ãʱȴ±í´ïÁ˶Է´Ïò´«²¥Ëã·¨µÄÖÊÒÉ¡£ËûÈÏΪ£¬·´Ïò´«²¥Ëã·¨²¢²»ÊÇ´óÄÔÔË×÷µÄ·½Ê½£¬ÒªÏëÈÃÉñ¾­ÍøÂç±äµÃ¸üÖÇÄÜ¡£¾Í±ØÐë·ÅÆúÕâÖÖΪËùÓÐÊý¾Ý±ê×¢µÄ·½Ê½¡£“ÎÒÃÇÐèÒª·ÅÆúµô·´Ïò´«²¥Ëã·¨£¨Backpropagation algorithm£©£¬ÖØпª±ÙÒ»Ìõеķ¾¶¡£”

ͼح½Ü¸¥Àï·ÐÁ¶Ù£¨Geoffrey Hinton£©

¾ÍÔÚ¼¸¸öСʱ֮ǰ£¬ÓÉ Hinton ºÍÆäÔڹȸè´óÄÔµÄͬÊ Sara Sabour¡¢Nicholas Frosst ºÏ×÷µÄ NIPS 2017 ÂÛÎÄ¡¶Dynamic Routing Between Capsules¡·ÒѾ­Õýʽ¶ÔÍ⹫¿ª£¬½âÊÍÁ˲»Í¬ Capsules ¼ä·ÓɵÄѧϰ¡£ºÁÎÞÒÉÎÊ£¬ÒÔ´ËΪÆðµã£¬È˹¤ÖÇÄÜδÀ´ÐÎ̬µÄÑݽøºÍËÜÔìºÜ¿ÉÄܽ«»á³ÊÏÖÒ»¸öÍêÈ«²»Ò»ÑùµÄÃæò¡£

ÂÛÎĵØÖ·£ºhttps://arxiv.org/abs/1710.09829

ͼح¸Õ¸Õ¹«²¼µÄHintonµÄÂÛÎÄ

ÔÚÕâƪÂÛÎÄÖУ¬Hinton Ê×ÏȽâÊÍ Capsule ×÷Ϊһ×éÉñ¾­Ôª£¬ËüµÄ»î¶¯ÏòÁ¿£¨activity vector£©±íʾÌض¨ÀàÐÍʵÌ壨Èçij¶ÔÏó»òij¶ÔÏóµÄ²¿·Ö£©µÄʵÀý»¯²ÎÊý¡£×÷ÕßʹÓûÏòÁ¿µÄ³¤¶ÈÀ´±íÕ÷ʵÌå´æÔڵĸÅÂÊÒÔ¼°ÆäʵÀý»¯²ÎÊýµÄ·½Ïò¡£¶ø»îÔ¾ÔÚͬһˮƽµÄ Capsule ͨ¹ý±ä»»¾ØÕó¶Ô¸ü¸ß¼¶±ð Capsule µÄʵÀý»¯²ÎÊý½øÐÐÔ¤²â¡£µ±¶à¸öÔ¤²âÏàͳһµÄʱºò£¬½Ï¸ß¼¶±ðµÄ Capsule ¾Í¿ªÊ¼±äµÃ»îÔ¾¡£

¾­¹ý×ܽᣬ¾­¹ýÇø±ðѵÁ·µÄ¶à²ã Capsule ϵͳÔÚ MNIST ÉÏʵÏÖÁË×îÓŵÄÐÔÄÜ£¬ÔÚʶ±ð¸ß¶ÈÖصþµÄÊý×Öʱ£¬ÆäЧ¹ûÒªÃ÷ÏÔºÃÓÚ¾í»ýÉñ¾­ÍøÂ硣ΪÁË´ïµ½ÕâÑùµÄ½á¹û£¬Hinton ËûÃÇʹÓÃÁ˵ü´úµÄ·ÓÉЭÒé»úÖÆ£¨routing-by-agreement mechanism£©£ºÒ»¸öµÍ²ãµÄ Capsule Ï£Íû½«ÆäÊä³ö·¢Ë͵½Ò»¸ö»î¶¯ÏòÁ¿¾ßÓдó±êÁ¿»ý¡¢Ô¤²âÀ´×ԵͲã Capsule µÄ¸ß²ã Capsule¡£

¶øÖ®ËùÒÔ Hinton »áÌá³ö Capsule µÄ¸ÅÄÕâÓëËû´Ó 80 Äê´ú¿ªÊ¼¾Í¹Ø×¢ÈËÀà´óÄÔµÄÔ­Àí²»ÎÞ¹Øϵ£¬Îª´ËËû»¹Ïà¼Ì·¢±íÁËÒ»¶¨ÊýÁ¿µÄÓйØÄÔÉñ¾­¿ÆѧµÄÂÛÎÄ¡£

ÔÚÈ˹¤ÖÇÄÜ·¢Õ¹µÄ³õÆÚ£¬°´ÕÕÕý³£µÄ˼·£¬Ñо¿ÈËÔ±»á×ÔÈ»¶øÈ»µÄÁªÏëµ½½«Î¨Ò»ÓµÓи߼¶Öǻ۵ÄÈËÄÔ×÷Ϊ»úÆ÷µÄÄ£Äâ¶ÔÏó£¬ÓÉ´ËÒ²¾Í³É¾ÍÁË»ðÈÈһʱµÄÈ˹¤Éñ¾­ÍøÂç¡£²»¹ý£¬µ¥²ãÈ˹¤Éñ¾­ÍøÂçµÄѵÁ··½Ê½ÉõÖÁÎÞ·¨Ê¹»úÆ÷ѧ»á×î¼òµ¥µÄ“Òì»ò”Âß¼­£¬¶à²ãÉñ¾­ÍøÂçµÄѵÁ·ÓÖÒ£Ò£ÎÞÆÚ¡£

×îÔçµÄÉñ¾­ÍøÂç Perceptron µ®ÉúÓÚ1960Äê´ú£¬±»ÓþΪÂõÏòÀàÈË»úÆ÷ÖÇÄܵĵÚÒ»²½¡£1969Ä꣬ÂéÊ¡Àí¹¤Ñ§ÔºµÄÂíÎÄ·Ã÷˹»ù£¨Marvin Minsky£©ºÍÎ÷Ħ·ÅÁ¶ûÌØ£¨Seymour Papert£©·¢±íÁËÖø×÷¡¶Perceptrons¡·£¬ÓÃÊýѧµÄ·½·¨Ö¤Ã÷ÕâÖÖÍøÂçÖ»ÄÜʵÏÖ×î»ù±¾µÄ¹¦ÄÜ¡£ÕâÖÖÍøÂçÖ»ÓÐÁ½²ãÉñ¾­Ôª£¬Ò»¸öÊäÈë²ãºÍÒ»¸öÊä³ö²ã¡£Èç¹ûÔÚÊäÈë²ãºÍÊä³ö²ãÖ®¼ä¼ÓÉϸü¶àµÄÍøÂ磬ÀíÂÛÉÏ¿ÉÒÔ½â¾ö´óÁ¿²»Í¬µÄÎÊÌ⣬µ«ÊÇûÈËÖªµÀÈçºÎѵÁ·ËüÃÇ£¬ËùÒÔÕâЩÉñ¾­ÍøÂçÔÚÓ¦ÓÃÁìÓòºÁÎÞ×÷ÓᣳýÁËÉÙÊýÏñ Hilton ÕâÑù²»·þÊäµÄÈË£¬´ó¶àÊýÈË¿´¹ýÕâ±¾Êéºó¶¼ÍêÈ«·ÅÆúÁËÉñ¾­ÍøÂçµÄÑо¿¡£

ͼح½Ü¸¥Àï·ÐÁ¶Ù£¨Geoffrey Hinton£©

ÓÚÊÇ£¬Hinton Ïëµ½ÁËÈ˹¤Éñ¾­ÍøÂç²»Ò»¶¨·ÇÒªÍêÈ«°´ÕÕÄ£ÄâÈËÄÔµÄģʽ¡£Òò´Ë£¬ÔÚÉÏÊÀ¼Í 80 Äê´ú£¬Hinton ÓÖÁí±Ùõ辶Ìá³öÁË¿ÉÒÔÓÃÀ´ÑµÁ·¶à²ãÉñ¾­ÍøÂçµÄ·´Ïò´«²¥Ëã·¨¡£×î¾ß±êÖ¾ÐԵľÍÊÇ 1986 Äê Hinton Óë David E. Rumelhart ºÍ Ronald J. Wlilliams ºÏÖøµÄ¡¶Learning representations by back-propagation errors¡·ÂÛÎĵķ¢±í¡£

2012 Ä꣬HiltonÓëËûÔÚ¶àÂ׶àµÄѧÉúÒ»µÀËù·¢±íµÄһƪÂÛÎıíÃ÷£¬Ó÷´Ïò´«²¥ÑµÁ·µÄÉî¶ÈÉñ¾­ÍøÂçÔÚͼÏñʶ±ðÁìÓò´ò°ÜÁ˵±Ê±×îÏȽøµÄϵͳ——“Éî¶Èѧϰ”ÖÕÓÚÃæÊÀ¡£ÔÚÍâ½ç¿´À´£¬È˹¤ÖÇÄÜËƺõÒ»Ò¹Ö®¼äͻȻ±¬·¢ÁË£¬µ«¶Ô Hilton ¶øÑÔ£¬ÕâÖ»ÊÇÒ»¸ö³Ùµ½µÄÀñÎï¡£

ͼح1986ÄêÐÁ¶ÙÓëËûÈ˺Ï×÷µÄÉñ¾­ÍøÂçÂÛÎÄ

µ«Ê±¼ä¹ýÈ¥ÁË 30 Ä꣬Hinton È´ÔÚÒ»¸ö¶àÔÂÇ°Ðû²¼·ÅÆúµôÁË·´Ïò´«²¥Ëã·¨¡£ÕâÊÇÒòΪ Hinton ʼÖÕÏàÐÅ¿Ë·þÈ˹¤ÖÇÄܾÖÏÞÐԵĹؼüÔÚÓڴ“Ò»¸öÁ¬½Ó¼ÆËã»ú¿ÆѧºÍÉúÎïѧµÄÇÅÁº”¡£´ÓÕâ¸ö½Ç¶È¿´£¬·´Ïò´«²¥ÊÇÊÜÉúÎïѧÆô·¢µÄ¼ÆËã»úѧͻÆÆ£»¸ÃÀíÄî×î³õ²¢·ÇÀ´×Ô¹¤³Ìѧ£¬¶øÊÇÀ´×ÔÐÄÀíѧ¡£Òò´Ë£¬Hinton Õý³¢ÊÔЧ·ÂÕâ¸öģʽ¡£

ʵ¼ÊÉÏ£¬Ä¿Ç°µÄÉñ¾­ÍøÂçÓÉ´óƽÃæ²ã×é³É£¬¼´Ã¿Ò»²ãµÄÉñ¾­Ôª¶¼×öµÄÊÇÀàËƵÄÊÂÇé¡£µ«ÈËÀàÐÂƤ²ãµÄÕæʵÉñ¾­Ôª²»½öÊÇˮƽ·Ö²¼³É²ãµÄ£¬»¹Óд¹Ö±ÅÅÁеġ£Hinton ÈÏΪ£¬ËûÖªµÀÕâЩ´¹Ö±½á¹¹µÄ×÷Ó㬱ÈÈçÔÚÈËÀàÊÓ¾õϵͳÖУ¬ÕâЩ½á¹¹È·±£ÎÒÃÇÔÚÊӽDZ仯ʱ±£³Öʶ±ðÎïÌåµÄÄÜÁ¦¡£Òò´Ë£¬Ëû´î½¨ÁËÃûΪ“Capsule”µÄÈ˹¤ÊÓ¾õÌåϵÀ´ÑéÖ¤Õâ¸öÀíÂÛ¡£

ÏÖÔÚ£¬¶Ô Capsule ÀíÂÛµÄÑо¿»¹´¦ÓڱȽÏÔçÆڵĽ׶Σ¬ÕâÒ²¾ÍÒâζ×ÅÆ仹ÓкܶàµÄÎÊÌâÓдý¿¼²ì¡£²»¹ý£¬ÏÖÔÚÒѾ­ÓÐÔ½À´Ô½¶àµÄ¼£Ïó±íÃ÷ Capsule ¿ÉÒÔ½â¾öһЩÎÊÌ⣬ÏàÐÅËüÊÇÒ»¸öÖµµÃ½øÒ»²½ÍÚ¾òµÄ·¾¶£¬ÕýÈç Hinton Ôø¶Ô¡¶ÂéÊ¡Àí¹¤¿Æ¼¼ÆÀÂÛ¡·±íʾ£¬“Capsule ÀíÂÛÒ»¶¨ÊǶԵģ¬²»³É¹¦Ö»ÊÇÔÝʱµÄ¡£”

Ïà¹ØÈÈ´ÊËÑË÷£ºÉî¶È ÂÛÎÄ

ÉÏһƪ£ºÏà¾àÒ£Ô¶µÄÐÇϵ֮¼ä¾¹È»Ò²»á½»»»ÎïÖÊ
ÏÂһƪ£º¸¨ÖúÉúÖ³¿Æ¼¼ÊÇ°ïÖúÈËÃÇʵÏÖÉúÓýÔ¸ÍûµÄ¿Æѧ

ҽѧÍƹã
ÈÈÃŹºÎï