From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on polar.synack.me X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham autolearn_force=no version=3.4.4 X-Google-Language: ENGLISH,ASCII-7-bit X-Google-Thread: 103376,2e8cf506f89b5d0a X-Google-Attributes: gid103376,public X-Google-ArrivalTime: 2001-09-12 20:58:52 PST Path: archiver1.google.com!newsfeed.google.com!newsfeed.stanford.edu!newsfeed.berkeley.edu!ucberkeley!enews.sgi.com!newshub2.rdc1.sfba.home.com!news.home.com!news1.rdc1.sfba.home.com.POSTED!not-for-mail From: tmoran@acm.org Newsgroups: comp.lang.ada Subject: Ada web crawler References: X-Newsreader: Tom's custom newsreader Message-ID: <%9Wn7.4328$L%5.3362526@news1.rdc1.sfba.home.com> Date: Thu, 13 Sep 2001 03:58:51 GMT NNTP-Posting-Host: 24.7.82.199 X-Complaints-To: abuse@home.net X-Trace: news1.rdc1.sfba.home.com 1000353531 24.7.82.199 (Wed, 12 Sep 2001 20:58:51 PDT) NNTP-Posting-Date: Wed, 12 Sep 2001 20:58:51 PDT Organization: Excite@Home - The Leader in Broadband http://home.com/faster Xref: archiver1.google.com comp.lang.ada:13063 Date: 2001-09-13T03:58:51+00:00 List-Id: David Botton has kindly posted finder.zip at www.adapower.com/os/finder.html It's source plus (Windows) executable for a program that crawls a site checking links. Thus finder www.adapower.com will scan the adapower site, following links to local html files and noting links to other files. finder www.adapower.com/os will scan just the "os" directory, treating any links outside that as "foreign", to be noted, but not scanned. Speed is of course highly dependent on internet access speed. The program is not polished, and still contains some capabilities that were needed for a specific application, but the source code is there for your customization.