Google, Yahoo Spiders Can Now Crawl Through Flash Sites

As anyone who has had the pleasure of doing web design and development through marketing agencies knows, Flash tends to be wildly popular among clients and wildly unpopular among, well, pretty much everyone else. Part of the reason for this is because Flash is so inherently un-Googleable; anything that goes into a Flash-only site is basically invisible to search engines and therefore, the world. That will no longer be the case, however, as Adobe announced today that it has teamed up with Google and Yahoo to make Flash files indexable by search engines.

This announcement has been a long time coming, as Flash developers have been wishing for ways to make their content searchable for close to a decade. Adobe acknowledges this in its announcement, saying that although search engines are able to index static text and links within Flash SWF files, “[Rich Internet Applications] and dynamic Web content have been generally difficult to fully expose to search engines because of their changing states—a problem also inherent in other RIA technologies.”

This announcement may also result in some major usability changes (for the better) for Flash on the web. In a post to its Webmaster Central Blog, Google wrote that it can now index all kinds of textual content in SWF files, like that included in Flash gadgets, buttons, menus, entirely self-contained Flash web sites, “and everything in between.” Google can now also follow URLs embedded within Flash files to add to the crawling pipeline. This new indexing technology does not, however, include FLV files (video files that are found on sites like YouTube) because those are generated as videos and don’t contain any text elements like an SWF file does.

Google says it’s able to do this by developing an algorithm that “explores Flash files in the same way that a person would,” by clicking buttons and manually going through Flash content. “Our algorithm remembers all of the text that it encounters along the way, and that content is then available to be indexed,” wrote the company. “We can’t tell you all of the proprietary details, but we can tell you that the algorithm’s effectiveness was improved by utilizing Adobe’s new Searchable SWF library.”

Of course, Google (and eventually Yahoo) won’t be able to index everything embedded within a Flash file—at least not yet. Anything that is image-related, including text that is embedded into images, will be invisible to the search engines for the time being. Google also noted that it can’t execute certain JavaScripts that may be embedded into a Flash file, and that while it indexes content that is contained in a separate HTML or XML file, it won’t be counted as part of the content in the Flash file. These are all issues that are being worked on, however, and are likely to change in the future.

Yahoo is also working with Adobe to index SWF files, but doesn’t appear to be as far along as Google just yet. One player that is noticeably missing is Microsoft, though. From Adobe’s announcement and the language used by Google, it appears as if each search engine has to work with Adobe to make this possible—meaning that Microsoft has either been excluded by Adobe for this round or has decided to voluntarily sit this one out. Either way, with searchable SWF files down, usability experts can now focus all of their attention on other Flash-related concerns, like blatant design perversion and excessive animation abuse.

Taken from http://arstechnica.com/news.ars/post/20080701-google-yahoo-spiders-can-now-crawl-through-flash-sites.html