[Home] [Setup] [Documentation] [Tools] [Toolbar] [Research]
Basic Configuration Name of the project Prefix
Directory to store the crawler
Path to youtube_dl (only required if you want to download flash videos)
No. of videos per query (up to 1000)
Database Setup Host name
Name of the database (should not already exist)
User name
Password
YouTube Setup
Indicate how often each of these items should be crawled    Never  Once  Every time

Title
Description
Username
Time when video added
Duration in seconds
Category
Keywords
Video-page URL
Thumbnail URL
Number of views
Number of ratings
Average rating
Number of comments
Number of video responses
Number of times favorited
Rank in the rank-list for a query

  • Make sure the directory that you choose to store the crawler in 'Basic Configuration' has recursive read+write access to your web-server.
  • While giving the directory paths in 'Basic Configuration', do not put '/' at the end.
  • Typical values for 'YouTube Setup' are already selected as the default.
  • What's the difference between 'once' and 'everytime'?
    'Once' attributes are recorded only the first time when the video is encountered by the crawler. You want to do this for the attributes that don't change over the time, such as the title of a video.
    'Everytime' attributes are recorded everytime the crawler visits a video. You want to do this for the attributes that keep changing, such as the number of videos for a video.
  • In 'Crawling Setup', use '*' as the value to ignore that parameter. For instance, if you want to run a process for every month of the year, put '*' in month. Once again, typical values are already given. Executing the queries and collecting 'once' attributes are scheduled to be executed at midnight (hour:0, min:0); collecting 'everytime' attributes are scheduled to be executed at 1am.
Crawling Setup
Event   Month (1-12)  Day of month (1-31)  Day of week (0-6)  Hour (0-23)  Min (0-59) 

Execute queries
Crawl
Download videos in Flash format
Convert videos to MPEG

| Send comments | Visit TubeKit Website | © 2007-2012 Chirag Shah |