ict_d2_serverroom
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
ict_d2_serverroom [2018/03/26 13:49] jpe |
ict_d2_serverroom [2019/07/12 15:23] (current) 143.169.217.228 [CDE D2 serverroom] |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| Almost all GPU servers (astra 2->9) are located in this room, where each server is provided whith redundant power supply and fast internet access and enough cool air. Recently (03/2018) the new storage server sto3 moved there as well and the configuration looks like this: | Almost all GPU servers (astra 2->9) are located in this room, where each server is provided whith redundant power supply and fast internet access and enough cool air. Recently (03/2018) the new storage server sto3 moved there as well and the configuration looks like this: | ||
| + | ^ rack ^ number ^ | ||
| + | | rack | ls.6.3 | | ||
| ^ device ^ rack location ^ | ^ device ^ rack location ^ | ||
| - | | netgear 10GB switch | U12 | | + | | Netgear 10GB switch | U17 | |
| - | | astra-tesla-9 | U10-11 | | + | | sto3 | U13-16 | |
| - | | sto3 | U5-9| | + | | astra-tesla-9 | U11-12| |
| - | | astra-tesla5-9 | U1-4 | | + | | astra-tesla5-8 | U7-10 | |
| + | | astra-tesla-4 | U5-6 | | ||
| + | | astra-tesla-3 | U3-4 | | ||
| + | | astra-tesla-2 | U1-2 | | ||
| + | |||
| + | Tesla-9 and sto3 are connected via 10GB ethernet NIC's and the Netgear switch. Later on we can connect the other tesla's as well to t his fast 10GB switch. This combination gives fast access to data stored on the storage server (e.g. uploaded data from partners), for calculations on the GPU server, as well as output data on the storage server, provided the given mount points over the 10GB switch are used. | ||
| - | Tesla-9 and sto3 are connected via 10GB ethernet NIC's and the Netgear switch. Later on we can connect the other tesla's as well to t his fast 10GB switch. | ||
| <code> | <code> | ||
| - | root@sto3:~# ethtool ens5f1 | + | root@sto3:~# ethtool ens5f1 | grep Speed |
| - | Settings for ens5f1: | + | |
| - | Supported ports: [ TP ] | + | |
| - | Supported link modes: 1000baseT/Full | + | |
| - | 10000baseT/Full | + | |
| - | Supported pause frame use: No | + | |
| - | Supports auto-negotiation: Yes | + | |
| - | Advertised link modes: 100baseT/Full | + | |
| - | 1000baseT/Full | + | |
| - | 10000baseT/Full | + | |
| - | Advertised pause frame use: No | + | |
| - | Advertised auto-negotiation: Yes | + | |
| Speed: 10000Mb/s | Speed: 10000Mb/s | ||
| - | Duplex: Full | ||
| - | Port: Twisted Pair | ||
| - | PHYAD: 0 | ||
| - | Transceiver: external | ||
| - | Auto-negotiation: on | ||
| - | MDI-X: Unknown | ||
| - | Supports Wake-on: d | ||
| - | Wake-on: d | ||
| - | Current message level: 0x00000007 (7) | ||
| - | drv probe link | ||
| - | Link detected: yes | ||
| - | | ||
| </code> | </code> | ||
| + | |||
| + | |||
| For astra-9 -> | For astra-9 -> | ||
| <code> | <code> | ||
| - | [vlab@astra-tesla-9 ~]$ ethtool ens1f1 | + | [vlab@astra-tesla-9 ~]$ sudo ethtool ens1f1 | grep Speed |
| - | Settings for ens1f1: | + | |
| - | Supported ports: [ TP ] | + | |
| - | Supported link modes: 1000baseT/Full | + | |
| - | 10000baseT/Full | + | |
| - | Supported pause frame use: No | + | |
| - | Supports auto-negotiation: Yes | + | |
| - | Advertised link modes: 100baseT/Full | + | |
| - | 1000baseT/Full | + | |
| - | 10000baseT/Full | + | |
| - | Advertised pause frame use: No | + | |
| - | Advertised auto-negotiation: Yes | + | |
| Speed: 10000Mb/s | Speed: 10000Mb/s | ||
| - | Duplex: Full | ||
| - | Port: Twisted Pair | ||
| - | PHYAD: 0 | ||
| - | Transceiver: external | ||
| - | Auto-negotiation: on | ||
| - | MDI-X: Unknown | ||
| - | Cannot get wake-on-lan settings: Operation not permitted | ||
| - | Current message level: 0x00000007 (7) | ||
| - | drv probe link | ||
| - | Link detected: yes | ||
| </code> | </code> | ||
| + | We use the 10.0.0.0/24 network for this fast interface. So if you want nfs to go via this interface, you have to specify the correct network interface. | ||
| + | ^ server ^ ipv4 ^ | ||
| + | | sto3 | 10.0.0.10 | | ||
| + | |astra-tesla-9 | 10.0.0.112 | | ||
| + | |astra-tesla-8 | 10.0.0.30 | | ||
| + | |astra-tesla-7 | 10.0.0.30 | | ||
| + | |astra-tesla-6 | 10.0.0.30 | | ||
| + | |astra-tesla-5 | 10.0.0.30 | | ||
| + | |astra-tesla-4 | 10.0.0.34 | | ||
| + | |astra-tesla-3 | 10.0.0.30 | | ||
| + | |astra-tesla-2 | 10.0.0.30 | | ||
| + | ====== 10GB switch ====== | ||
| + | |||
| + | To use these 10GB NIC cards, we need a switch with the same capacity. -> | ||
| + | Netgear XS716E-100NES , referenced in this {{ :wiki:inv55464.pdf |invoice}} \\ | ||
| + | ** TODO** | ||
| + | This switch is a managed switch, and has MAC address 9C:3D:CF:F6:F6:1A. We still need to assign a proper ip address to gain access to this switch. | ||
| + | |||
| + | We have to find out how we can connect the astra-tesla-5-9 with this switch. This modular server maybe doesn't the possibility to extend it with a second NIC card. | ||
| + | |||
ict_d2_serverroom.1522072185.txt.gz · Last modified: 2018/03/26 13:49 by jpe
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
