SSH Tunneling with restricted/limited shell

If you want to connect to rsync/mysql or other service via localhost through SSH tunneling, you need the tunnel to stay open. There are rssh and other limited shells. But you can write your own. In this case, it opens for 10 seconds then closes. It does not close if there are activities (rsync takes longer than 10s) from my experience. It does nothing so there is no security to worry about.

1) Create sleepshell.c

main() {sleep (10);}

2) gcc sleepshell.c -o /usr/local/bin/sleepshell

3) useradd -m -d /home/tunneluser -s /usr/local/bin/sleepshell tunneluser

or usermod -s /usr/local/bin/sleepshell tunneluser

And that is it.

ssh -f -L 873:localhost:873 tunneluser@server “ANYTHING”

rsync –delete –delete-excluded –progress -zrltvp module@localhost dest

Steadfast Cloud

These are some benchmarks run with UnixBench 5.1.3 on Steadfast Cloud powered by OnApp. The ability for scaling up/down RAM and CPU without reboot is neat and very useful. Disk resizing need reboot but this can be avoided by the app. The OnApp API is very simple and easy to use as well. I tried RackSpace and it needs reboot. EC2 is similar to RackSpace cloud too. Basically, migration to a new instance (bigger/smaller) and point to the same mount/virtual disk. OnApp added memory and CPU in real-time and I can see it reflect via “top”. Very cool indeed. I see this as the future (auto-scaling up/down, in/out, vertical/horizontal seamlessly) and truely present the power of cloud computing

Physical: 2x4xHT (16 cores) Xeon E5620 2.4Ghz

Config – 512MB RAM, 10% of 1 core and also 1% of 1 core (look like it’s burst-able)

Benchmark Run: Wed Dec 07 2011 20:46:04 – 21:14:18
1 CPU in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       14152617.2 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     2851.9 MWIPS (9.9 s, 7 samples)
Execl Throughput                               1588.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        215988.3 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           56996.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        651739.5 KBps  (30.0 s, 2 samples)
Pipe Throughput                              290994.0 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  83294.7 lps   (10.0 s, 7 samples)
Process Creation                               3397.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   2741.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                    381.3 lpm   (60.0 s, 2 samples)
System Call Overhead                         251441.4 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   14152617.2   1212.7
Double-Precision Whetstone                       55.0       2851.9    518.5
Execl Throughput                                 43.0       1588.2    369.4
File Copy 1024 bufsize 2000 maxblocks          3960.0     215988.3    545.4
File Copy 256 bufsize 500 maxblocks            1655.0      56996.5    344.4
File Copy 4096 bufsize 8000 maxblocks          5800.0     651739.5   1123.7
Pipe Throughput                               12440.0     290994.0    233.9
Pipe-based Context Switching                   4000.0      83294.7    208.2
Process Creation                                126.0       3397.9    269.7
Shell Scripts (1 concurrent)                     42.4       2741.2    646.5
Shell Scripts (8 concurrent)                      6.0        381.3    635.5
System Call Overhead                          15000.0     251441.4    167.6
========
System Benchmarks Index Score                                         433.8

Config – 512M RAM, 1% of 10 core

Benchmark Run: Wed Dec 07 2011 22:05:17 – 22:33:32
10 CPUs in system; running 1 parallel copy of tests

Dhrystone 2 using register variables       13349714.7 lps   (10.0 s, 7 samples)
Double-Precision Whetstone                     2824.6 MWIPS (9.9 s, 7 samples)
Execl Throughput                               1241.2 lps   (30.0 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        213946.6 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           56376.5 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        660924.3 KBps  (30.0 s, 2 samples)
Pipe Throughput                              301494.5 lps   (10.0 s, 7 samples)
Pipe-based Context Switching                  80504.0 lps   (10.0 s, 7 samples)
Process Creation                               3496.9 lps   (30.0 s, 2 samples)
Shell Scripts (1 concurrent)                   3429.2 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   1309.2 lpm   (60.0 s, 2 samples)
System Call Overhead                         261392.6 lps   (10.0 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   13349714.7   1143.9
Double-Precision Whetstone                       55.0       2824.6    513.6
Execl Throughput                                 43.0       1241.2    288.7
File Copy 1024 bufsize 2000 maxblocks          3960.0     213946.6    540.3
File Copy 256 bufsize 500 maxblocks            1655.0      56376.5    340.6
File Copy 4096 bufsize 8000 maxblocks          5800.0     660924.3   1139.5
Pipe Throughput                               12440.0     301494.5    242.4
Pipe-based Context Switching                   4000.0      80504.0    201.3
Process Creation                                126.0       3496.9    277.5
Shell Scripts (1 concurrent)                     42.4       3429.2    808.8
Shell Scripts (8 concurrent)                      6.0       1309.2   2182.0
System Call Overhead                          15000.0     261392.6    174.3
========
System Benchmarks Index Score                                         479.6

————————————————————————
Benchmark Run: Wed Dec 07 2011 22:33:32 – 23:02:22
10 CPUs in system; running 10 parallel copies of tests

Dhrystone 2 using register variables       88693187.8 lps   (10.1 s, 7 samples)
Double-Precision Whetstone                    24944.7 MWIPS (9.8 s, 7 samples)
Execl Throughput                               8738.4 lps   (29.8 s, 2 samples)
File Copy 1024 bufsize 2000 maxblocks        138675.4 KBps  (30.0 s, 2 samples)
File Copy 256 bufsize 500 maxblocks           36445.3 KBps  (30.0 s, 2 samples)
File Copy 4096 bufsize 8000 maxblocks        485151.9 KBps  (30.0 s, 2 samples)
Pipe Throughput                             2266234.2 lps   (10.1 s, 7 samples)
Pipe-based Context Switching                 618203.4 lps   (10.1 s, 7 samples)
Process Creation                              20493.0 lps   (30.1 s, 2 samples)
Shell Scripts (1 concurrent)                  15570.8 lpm   (60.0 s, 2 samples)
Shell Scripts (8 concurrent)                   2282.2 lpm   (60.2 s, 2 samples)
System Call Overhead                        1834019.4 lps   (10.1 s, 7 samples)

System Benchmarks Index Values               BASELINE       RESULT    INDEX
Dhrystone 2 using register variables         116700.0   88693187.8   7600.1
Double-Precision Whetstone                       55.0      24944.7   4535.4
Execl Throughput                                 43.0       8738.4   2032.2
File Copy 1024 bufsize 2000 maxblocks          3960.0     138675.4    350.2
File Copy 256 bufsize 500 maxblocks            1655.0      36445.3    220.2
File Copy 4096 bufsize 8000 maxblocks          5800.0     485151.9    836.5
Pipe Throughput                               12440.0    2266234.2   1821.7
Pipe-based Context Switching                   4000.0     618203.4   1545.5
Process Creation                                126.0      20493.0   1626.4
Shell Scripts (1 concurrent)                     42.4      15570.8   3672.3
Shell Scripts (8 concurrent)                      6.0       2282.2   3803.6
System Call Overhead                          15000.0    1834019.4   1222.7
========
System Benchmarks Index Score                                        1630.6

 

Some autobench results:

1% of 1 cores => ab @ 450 reqs/sec
1% of 2 cores => ab @ 940 reqs/sec
1% of 10 cores => ab @ 1060 reqs/sec
2% of 5 cores => ab @ 1005 reqs/sec
5% of 2 cores => ab @ 990 reqs/sec

RackSpace cloud resizing (Scaling) steps

Step 1: Resize request is queued
The resize request will be processed as resources permit. The queued period is usually quick, but can vary depending on other host machine activities.
Step 2: Initial preparation of new server
This is the longest part of the upgrade, during this step the original server remains up and running.
Step 3: Current server is shut down and migrated
You will experience a downtime varying in proportion to fullness of the filesystem.
Step 4: Verification of new server
Your newly resized server should now be available. Login to verify that the server is intact and functioning as expected. After verification, the old server will be purged from the system and your account will be billed a prorated amount. If no verification is made, the resize will be automatically verified after a period of 12 hours.

Benchmark xtrabackup streaming option tar.gz vs. tar

time innobackupex –databases=”mydb” –stream=tar ./ –slave-info > /var/lib/mysql/snapshot_mydb.tar

time innobackupex –databases=”mydb” –stream=tar ./ –slave-info | gzip – > /var/lib/mysql/snapshot_mydb.tar.gz

Results:
TAR: filesize 5.4Gb
real 0m52.633s
user 0m0.900s
sys 0m29.860s

TAR.GZ: filesize 350Mb
real 2m26.606s
user 3m40.261s
sys 0m28.829s

And during the dump, tar.gz locks tables. Thus, for a big database, a better solution would be just tar, then compress before sending

OSSEC vs. SVN

If you found the server suddenly blocks access when you try to “svn commit”, it might be because of OSSEC active response, which is good but a false alarm in this case (too many 404). Solution: many, but mine is just simply turn OSSEC off while committing and back on later. I don’t rename files that often.

http://www.ourada.org/blog/archives/188

Puppet, bittorrent, aria2

Bittorrent package for Linux is getting old (4.4 from RPMForge and 5.x from the source). Therefore, I made the switch to aria2 (command line is “aria2c”), which is much much better and offer a lot more features. Here is what I have for aria2.conf

daemon
enable-dht=true
allow-overwrite=true
seed-ratio=2
max-upload-limit=600K
bt-min-crypto-level=arc4
bt-require-crypto=true

Here is how to run it:

/usr/bin/aria2c –conf-path=aria2.conf abc.torrent >> torrent.log

Here is the old way running bittorrent via puppet:

 

exec { “run-torrents”:
command => ‘nohup launchmany-console –saveas_style 1 –max_upload_rate 600 –display_interval 5 . > torrent.log >/dev/null 2>&1 &’,
cwd => ‘/home/mytorrents’,
unless => ‘ps -e | grep launchmany’,
require => [ Package[‘bittorrent’] , Exec[‘download-torrents’] ]
}

Autobench, httperf – Web Server Benchmarking

Automate the benchmarking process for new/existing server:

# need to install autobench and httperf
autobench –single_host –host1 localhost –uri1 /stresstest/simple.php –low_rate 1000 –high_rate 10000 –rate_step 100 –const_test_time 20 –timeout 5 –quiet –file results.tsv

# graph the results in PostScript file (Mac Mail can preview this automatically)
rm -f results.ps && bench2graph results.tsv results.ps 4 5 6 7 8

# send the graph via email for simplicity (“mutt” can send email attachment via command line)
mutt -s “Autobench – Graph Results” -a results.ps root@localhost < results.tsv

Door-to-door is hard

Our neighborhood is organizing a petition to fight the city against the plan to convert a golf course, open space to more housing. Knocking door-to-door to get signature is hard. Some people don’t know about the issue (even with many reminders, letters sent to them). Likely just threw away, don’t care, don’t read, or don’t know English. One of the letters even has Spanish and Vietnamese so there is no excuse for ignorance. Some people don’t want to talk to us. Some people think we’re trying to sell them something. It’s my first time as an community activist and organizer so these downers just really suck. But there are also other people who really appreciate our volunteering efforts. That’s a social issue and people from all walk of life, some that you don’t want to bump into.

Good garden deals

These are some of the deals that are seasonal and the timing is very important. It’s a reminder for myself to wait until these times to get great deals

  • Playsand 50lbs for $2 (Lowes, around June/July)
  • 75% off plastic planter/pot/container (16″ for $3.24) at RiteAid (July 20, ad run from about July 15 – Aug 15)
  • Seeds (20% off), plants (40% off) (SummerWinds Nursery, end Sep, begin/mid Oct)

shell_exec silent error

This is likely a common error yet might be hard to detect. If you run shell_exec(), make sure to use the full path to the binary file. It might be obvious but many for convenience does not do so, including myself at times. You forget that .bashrc does these for you while running PHP script via crontab is entirely different.

For example, shell_exec(‘ifconfig’) returned empty (it happened at times, on different servers, intermittently, not always, that made thing even harder to understand). However, the path was the issue since the logic only went through in certain cases and it worked 2/4 servers. Weird! And if you run the script directly, it gets your path. Running it via cron does not have that luxury so you need the full path: shell_exec(‘/sbin/ifconfig’); Oh well, lesson learned.