[JavaScript] 使用 css selector 來撰寫爬蟲程式

過去習慣在寫爬蟲程式時都是使用正規表示法來剖析網頁結構取得資料。

最近在寫 Chrome Extension 時 Chrome Extension 可以使用 chrome.tabs.executeScript 在一個 tab 內的內容執行自己寫的 JavaScript，在這個機制可以利用 Css Selector 來做網頁的剖析取得想要的資料，極其方便。

所以一個簡單的想法是，取得網頁資料->執行 JavaScript ->取得想要的資料

就找到 Node.js 有個 Project jsdom 符合此需求

此程式便是使用 jsom 的去 ted 網站取得演講資訊
完整的 Source

var jsdom = require('jsdom');
jsdom.env ( "http://tedxtaipei.com/talks/2013-ping-cheng-yeh/" ,
            ["//code.jquery.com/jquery-1.11.3.min.js"] ,
            function(err, window) {
                var $ = window.jQuery;
                var speech = {
                    title : $('.textover .title').text(),
                    description : $('.content-main .section p').text(),
                    video : $('.player .placeholder').attr('data-embed-code').match(/src="(.*?)"/i)[1],
                    speaker : {
                        name : $('.speaker .name text').text(),
                        anotherName : $('.speaker .another-name text').text(),
                        jobTitle : $('.speaker .job-title').text(),
                        image : $('.speaker .cover .image').attr('style').match(/url\((.*)\)/i)[1],
                        description : $('.speaker .description .content p').text()
                    }
               };
               console.log(speech);
}); 

題外話
php 可以用 DOMDocument

Reference：
jsdom

[CSS] CSS 選擇器

筆記起來...
ref: http://www.w3.org/TR/CSS2/selector.html

Pattern	Meaning	Described in section
*	Matches any element.	Universal selector
E	Matches any E element (i.e., an element of type E).	Type selectors
E F	Matches any F element that is a descendant of an E element.	Descendant selectors
E > F	Matches any F element that is a child of an element E.	Child selectors
E:first-child	Matches element E when E is the first child of its parent.	The :first-child pseudo-class
E:link E:visited	Matches element E if E is the source anchor of a hyperlink of which the target is not yet visited (:link) or already visited (:visited).	The link pseudo-classes
E:active E:hover E:focus	Matches E during certain user actions.	The dynamic pseudo-classes
E:lang(c)	Matches element of type E if it is in (human) language c (the document language specifies how language is determined).	The :lang() pseudo-class
E + F	Matches any F element immediately preceded by a sibling element E.	Adjacent selectors
E[foo]	Matches any E element with the "foo" attribute set (whatever the value).	Attribute selectors
E[foo="warning"]	Matches any E element whose "foo" attribute value is exactly equal to "warning".	Attribute selectors
E[foo~="warning"]	Matches any E element whose "foo" attribute value is a list of space-separated values, one of which is exactly equal to "warning".	Attribute selectors
E[lang|="en"]	Matches any E element whose "lang" attribute has a hyphen-separated list of values beginning (from the left) with "en".	Attribute selectors
DIV.warning	Language specific. (In HTML, the same as DIV[class~="warning"].)	Class selectors
E#myid	Matches any E element with ID equal to "myid".	ID selectors