rsync file transfer speed

[adizero]

mga bro

centos 6 nfs mounted to QNAP nas.

kapag nagrsync kami nagbabago bago ung speed.  kaya kapag complete na ung rsync ang average nya is 100MB/s lang

example   sa umpisa  247MB/s sya  then babab and tataas ulet

time rsync -Havessh --progress --stats -h 10G-file.txt /data/maps/test-transfer/
sending incremental file list
10G-file.txt
259.72M   2%  247.66MB/s    0:00:40

after mga ilang secs,  dropped to 4MB/s

time rsync -Havessh --progress --stats -h 10G-file.txt /data/maps/test-transfer/
sending incremental file list
10G-file.txt
3.18G  30%    4.23MB/s    0:28:12


tapos tataas ulet sa 150  then 240 

pero in the end,     nag average sya ng 100 MB/s          hindi ba pwedeng  fixed na 240MB/s  ito ?

nagtry kami gumamit ng DLink unmanage switch and ARISTA manage switch pero same lang ang result




time rsync -avzmP --stats -h 10G-file.txt /data/maps/test-transfer/
building file list ...
1 file to consider
10G-file.txt
      10.51G 100%  104.53MB/s    0:01:35 (xfer#1, to-check=0/1)

Number of files: 1
Number of files transferred: 1
Total file size: 10.51G bytes
Total transferred file size: 10.51G bytes
Literal data: 10.51G bytes
Matched data: 0 bytes
File list size: 32
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 10.22M
Total bytes received: 31

sent 10.22M bytes  received 31 bytes  95.97K bytes/sec
total size is 10.51G  speedup is 1028.35

[splerdu]

What's the sustained write speed of your NAS? What drives do you have inside? How is the NAS connected to the network?

100MB/s is peak transfer rate for a 1GbE connection. You won't get more than that unless you can team up more links or upgrade your network to 10GbE.

As for your initial 240MB/s, that's likely because of an initial cache or buffer.

1. 240MB/s cached transfer
2. 4MB/s buffer full, flushing
3. 100MB/s actual sustained transfer speed

Check out SmallNetBuilder's NAS benchmarks. Notice how everything is around 100MB/s
http://www.smallnetbuilder.com/tools/charts/nas/view

[adizero]

PC drive is SSD  = >   QNAP drive is 3TB 7200RPM  =  100 MB/s

From NetAPP = > PC => QNAP    30MB/s lang    (both Netapp and QNAP are mounted as NFS on PC)


Nagtry kami mag network bond.  3pcs   1GB NIC's     lahat ng mode same lang result. 

QNAP naka trunk mode  Round robin, tlb  , 802.3ad   


pero nag dropped ung transfer rate to 30MB/s   from NetAPP  SATA to   QNAP 7200 RPM


magbabago ba tlga based sa source disk and destination disk speed ?  it does not matter kung 10Gbps switch pa ang gamit mo?


Ang goal kasi namin is to transfer files from Netapp  to QNAP ng mabilis.

ung 1.5TB  inaabot ng whole day bago matapos.   meron bang way mapabilis ito?

[bravoexo]

Read Speed > Write Speed?

[splerdu]

4-bay QNAP will go 400MB/s with 4x1GbE ports bonded
https://www.qnap.com/en-us/product_x_performance/product.php?II=149

How many drives do you have installed? Usually 100-150MB/s average write speed per HDD


If 1-bay NAS it'll max out at 80-100MB/s
https://www.qnap.com/en-us/product_x_performance/product.php?type=1&II=8

[adizero]

Gamit namin is  HS251+    https://www.qnap.com/en-in/product/model.php?II=197

2bay sya pero   1pc 3TB 7200rpm lang nakalagay.     Kapag ginawa namain 2pcs 3TB  marereach namin ang at least  100MB/s ?

nagtaka lang kami sa  setup below. bakit 30MB/s lang    kahit na 7200rpm ang Netapp and QNAP.

pero kapag ang file ay galing sa PC 100MB/s + sya     ( PC and QNAP  naka set mtu 9000)

From NetAPP = > PC (SSD & 3GB NIC's) => QNAP    30MB/s lang    (both Netapp and QNAP are mounted as NFS on PC)

[barurutor]

Your netapp is probably the bottleneck.

Try transfer from netapp to PC, does it reach 100MB/s?

[adizero]

NetAPP =NFS> PC root / SSD  =  100 MB/s