Crawling web
We have specific virtual machines that can be used for crawling. Feel free to use them.
Please note that only silk04-07 are currently available.
| Server | OS | Memory | Arch | Cores | Login | Description |
|---|---|---|---|---|---|---|
| silk04.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk05.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk06.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk07.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk08.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk09.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk10.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk11.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk12.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk13.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk14.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk15.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk16.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk17.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk18.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk19.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk20.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk21.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk22.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk23.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk24.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |
| silk25.stanford.edu | Ubuntu 16.04 | 8 GB | 64 bit | 1 | CSID | Crawling server |