FAQ

What is DomainWebCrawler?

  • DomainWebCrawler is a website analyzing engine that gathers data and generates reports.

    Why is it useful?

  • DomainWebCrawler collects various data in a few seconds gathering information and producing quick analysys of it. If you are a web-developer you may find it practical and useful.

    Why register?

  • It is a VERY fast procedure and after finishing it you can analyse your own site (or any other) and leave comments.

    What kind of information does DomainWebCrawler provide me with?

    • meta information (title/description/keywords)
    • html tags. DomainWebCrawler is working with all tags including user described
    • links and anchors
    • HTTP header
    • robots.txt
    • IP address
    • geo location @ Google Maps
    • Alexa trafic graph
    • tag cloud

    Site adding fails, I am redirected to this FAQ!

    • make sure you are logged in - there must be a greeting like "Welcome username" on the sidebar
    • make sure the domain name is spelled correctly
    • the index page must have unique title
    • the index page size should be bigger than 8 KB
    • the analyzed content cannot include offensive and "adult" words and expressions

    I want to remove my site fromDomainWebCrawler.

  • Add these lines to robots.txt:
    User-agent: truewebreport
    Disallow: /

  • Submit your domain here:

    What is Keyword Tag Analysis?

  • It is the number of appearances of keywords in HTML tags.

    What is Advanced Search?

  • You can use these special operators for smarter search:
    + plus sign tells that this word must be present in site report.
    - minus sign tells that this word must NOT be present in site report.
    * asterisk is a wildcard. It can be appended to the first letters of the word.
    NB! in multiple word searches there is OR by default, not AND, so it will return all sites that contain ANY of the words you inserted.
     Advanced

    I don't see some data (map/graph/meta)!

  • Some pages don't have meta, some servers cannot be identified precisely enough to show them on the map and some fresh or low-traffic sites are not in Alexa.com database yet.

    Are you going to upgrade DomainWebCrawler?

  • Sure. If you got some ideas - email us!

    I have a question/idea/found a bug. How can I get in touch with you?

  • We appreciate any feedback. Our email is: truewebreport -at- gmail -dot- com.