Mikrotik: routeros and siera U320
I have just bought a Siera Wireless 320U (from Telstra) and have had good success with it in a RB751G-2HnD running routeros 5.20. I am not in a LTE coverage area so have not tried this in Direct IP/LTE mode but it works fine for 3G. When you plug the 320U into the routerboard it should appear as `USB2.
- Create a new PPP session
- Select “usb2”
- set “Data Channel” to 3
- set “Info Channel to 4.
- set APN to “telstra.internet” (or whatever is appropriate for your plan/carrier)
- set the username and password
- if you have a pin set on your SIM then set this too (I have not tested this with a PIN)
- Under the PPP tab, make sure to de-select “Dial On Demand”
- Leave all the other settings as default.
You should now have a new interface that can connect to 3G. Don’t forget to setup NAT for this new interface.
Use your Samsung Galaxy 4S with AWS using SSH
Just switched from my iPhone 4s to != iPhone5.
So first (geek) question is:
how do I telnet?
The somewhat unexpected answer to me was Telnet IDE. It is a very natural implementation of a terminal. You just get a reduced shell on your android, but the really nice feature is the keyboard. You enable the keyboard under Settings->My Device->Language and Input. Just select Terminal IDE as your input and you will get all the keys you are missing. Like alt, arrows, ctrl etc.
So… now if you have your free instance in Amazon that I wrote about earlier. You want to ssh to that instance with your android? It turned out to be a bit of work….
I think this is the easy (ish) way.
Since you already have 50 gig on dropbox (if you registered your galaxy), might as well use it.
If you are using ubuntu in AWS, go ahead and add your dropbox repository with your normal way of ssh’ing to it:
create /etc/apt/sources.list/dropbox.list
with the following:
deb http://linux.dropbox.com/ubuntu precise main
do the usual with apt-get update etc… apt-get install dropbox
oh and do install the key that get printed out as well:)
apt-key adv --keyserver php.mit.edu --recv-keys <key>
when you do dropbox start it does want you to add the host in dropbox. You can do that on your local laptop since you probably don’t have X installed in the AWS cloud.
Restart dropbox and you should have a Dropbox directory replicating your data!
Now you can get stuff from your phone to the AWS instance without effort:)
so we need a ssh key to use with amazon. To create the key on your android:
open a terminal in TerminalIDE and:
dropbearkey -t rsa -f ~/.ssh/id_rsa
Now you have the key. Let’s add it to AWS.
Generate the ascii version of the key
dropbearkey -y -f ~/.ssh/id_rsa >/storage/sdcard0/files/id_rsa.txt
Now you can upload it with your dropbox app on the phone. Go to upload and browse to “Other Files”->”Internal Storage”->”files”
So now go ahead and ssh to your aws host. The file should be in your ~/Dropbox/ directory. Then just
cat id_rsa.txt >>~/.ssh/authorized_keys
Now you can ssh with telnetIDE:
ssh ubuntu@hostname.inaws.com -i id_rsa
Playing with FreeIPA and puppet
So I just rolled a new vm to hack around with FreeIPA. Here are some things that I’ve come across so far. I was planning on configuring LDAP, and Kerberos manually, but the included webui looks like a lovely tool to have for the data entry, user administrator type who likes to click on things. Let’s explore…
/etc/hosts:
FreeIPA is choosy about how your /etc/hosts is formatted. It requires an entry that has a particular order, that is:
192.168.123.7 ipa.example.com ipa
Obviously replace with your own values. This presents itself as:
The host name ipa.example.com does not match the primary host name ipa. Please check /etc/hosts or DNS name resolution
I had to dive into the source to figure this one out!
webui:
I’m in hack mode, and my laptop (hack station) is not participating in the domain that I’m pretending to manage. In addition, I’m not directly connected to the vm where I’m testing out FreeIPA. As usual, I port forward:
$ ssh root@ipa -L 1443:localhost:443
but when attempting to try the webui:
$ firefox https://localhost:1443/ipa/ui/
I get redirected to the official fqdn, and at port 443. After searching around, it turns out there is a: –no-ui-redirect option that you can pass to the ipa-server-install program, but it only comments out one line of the /etc/httpd/conf.d/ipa-rewrite.conf and doesn’t let me do exactly what I want. I’m sure someone with pro apache skills could hack this up properly, but I haven’t the patience.
As user ab in #freeipa kindly pointed out:
01:21 < ab> primary authentication method of web ui is Kerberos. Your browser, when configured, will need to obtain a kerberos ticket against web ui's server, that's why you're forced to connect to fully qualified hostname 01:22 < ab> otherwise a browser would attempt to obtain ticket to HTTP/localhost@REALM which does not exist 01:22 < ab> and wouldn't be what web server web ui is running on is using
which is a good point. For hacking purposes, I’m happy to forgo kerberos logins and type in a password manually, but since my use case is particularly small, I’ll just hack around this for now, and maybe a future FreeIPA will get this option. At the moment, it’s not supported.
A bad hacker could modify their /etc/hosts to include:
127.0.0.1 ipa.example.com ipa localhost.localdomain localhost
and run ssh as root (very bad!):
$ sudo ssh root@ipa -L 443:localhost:443 -L 80:localhost:80
to get easily get access locally. But don’t do this. It’s evil.
inactivity timeouts:
The web ui times out after 20 minutes. To increase this add:
session_auth_duration=42 minutes
to your /etc/ipa/default.conf, and restart httpd. You can have a look at the parser for an idea of what kind of values are acceptable.
puppet?:
As you might agree, it’s nice to have puppet modules to get you up and running. FreeIPA was easy to install, and my puppet module now makes it automatic. I’ve written a lot of fancy puppet code to manage your IPA resources. It’s not quite finished, and more resource types are on the way, but you can follow along at:
https://github.com/purpleidea/puppet-ipa
Happy hacking,
James
Made businessweek (rubicon that is:))
Nice press-release by AMD on what we done together on the Seamicro platform.
puppet lsi hardware raid module
In response to some discussion in the gluster community, I am releasing my puppet-lsi module. It’s quite simple, but it is very useful for rebuilding machines. It could do a lot more, but I wanted to depend on the proprietary LSI tools as little as possible. Running megacli with puppet would be a very doable hack, but I’m not sure enough devops out there who would use that feature.
Usage is straightforward if you like the sensible defaults:
include lsi::msm
The general idea is that you’ve probably already setup all your “virtual drive” RAID configurations initially, and now you’re deploying your setup using cobbler and puppet-gluster. This puppet-lsi module should install all the client side LSI tools, and make sure monitoring for the hardware RAID is working. Megacli and all the (evil?) vivaldi framework stuff will be up and running after puppet has run.
I haven’t tested this on a wide array of hardware, and there might even be newer LSI gear on the market. Please don’t test it on production servers. If you want help with this, you might have to sponsor some hardware, or send me somewhere where I can hack on some, because I don’t have a gluster test rig at the moment.
I am curious to hear what kind of RAID you’re using with gluster. Hardware? Software? Details rock. SGPIO with mdadm, and you’re my hero. I want to hear about that!
https://github.com/purpleidea/puppet-lsi/
I hope this was useful to you, and in the meantime,
Happy hacking,
James
PS: The most useful feature of this module, is that it sets up monitoring of your RAID, and lets you access the management daemon through the now installed LSI services.
Introduction to MCollective deck
I’ve not had a good introduction to MCollective slide deck ever, I usually just give demos and talk through it. I was invited to talk in San Francisco about MCollective so made a new deck for this talk.
On the night I gave people the choice of talks between the new Introduction talk and the older Managing Puppet using MCollective and sadly the intro talk lost out.
Last night the excellent people at Workday flew me to Dublin to talk to the local DevOps group there and this group was predominantly Chef users who chose the Introduction talk so I finally had a chance to deliver it. This talk was recorded, hopefully it’ll be up soon and I’ll link to it once available.
This slide deck is a work in progress, it’s clear I need to add some more information about the non-cli orientated uses of MCollective but it’s good to finally have a deck that’s receiving good feedback.
We uploaded the slides back when I was in San Francisco to slideshare and those are the ones you see here.
Collecting duplicate resources in puppet
I could probably write a long design article explaining why identical duplicate resources should be allowed [1] in puppet. If puppet is going to survive in the long-term, they will have to build in this feature. In the short-term, I will have to hack around deficiency. As luck would have it, Mr. Bode has already written part one of the hack: ensure_resource.
Why?
Suppose you have a given infrastructure with N vaguely identical nodes. N could equal 2 for a dual primary or active-passive cluster, or N could be greater than 2 for a more elaborate N-ary cluster. It is sufficient to say, that each of those N nodes might export an identical puppet resource which one (or many) clients might need to collect, to operate correctly. It’s important that each node export this, so that there is no single point of failure if one or more of the cluster nodes goes missing.
How?
As I mentioned, ensure_resources is a good enough hack to start. Here’s how you take an existing resource, and make it duplicate friendly. Take for example, the bulk of my dhcp::subnet resource:
define dhcp::subnet(
$subnet,
# [...]
$range = [],
$allow_duplicates = false
) {
if $allow_duplicates { # a non empty string is also a true
# allow the user to specify a specific split string to use...
$c = type($allow_duplicates) ? {
'string' => "${allow_duplicates}",
default => '#',
}
if "${c}" == '' {
fail('Split character(s) cannot be empty!')
}
# split into $realname-$uid where $realname can contain split chars
$realname = inline_template("<%= name.rindex('${c}').nil?? name : name.slice(0, name.rindex('${c}')) %>")
$uid = inline_template("<%= name.rindex('${c}').nil?? '' : name.slice(name.rindex('${c}')+'${c}'.length, name.length-name.rindex('${c}')-'${c}'.length) %>")
$params = { # this must use all the args as listed above...
'subnet' => $subnet,
# [...]
'range' => $range,
# NOTE: don't include the allow_duplicates flag...
}
ensure_resource('dhcp::subnet', "${realname}", $params)
} else { # body of the actual resource...
# BUG: lol: https://projects.puppetlabs.com/issues/15813
$valid_range = type($range) ? {
'array' => $range,
default => [$range],
}
# the templating part of the module...
frag { "/etc/dhcp/subnets.d/${name}.subnet.frag":
content => template('dhcp/subnet.frag.erb'),
}
}
}
As you can see, I added an $allow_duplicates parameter to my resource. If it is set to true, then when the resource is defined, it parses out a trailing #comment from the $namevar. This can guarantee uniqueness for the $name (if they happen to be on the same node) but more importantly, it can guarantee uniqueness on a collector, where you will otherwise be unable to workaround the $name collision.
This is how you use this on one of the exporting nodes:
@@dhcp::subnet { "dmz#${hostname}":
subnet => ...,
range => [...],
allow_duplicates => '#',
}
and on the collector:
Dhcp::Subnet <<| tag == 'dhcp' and title != "${dhcp_zone}" |>> {
}
There are a few things to notice:
- The $allow_duplicates argument can be set to true (a boolean), or to any string. If you pick a string, then that will be used to “split” out the end comment. It’s smart enough to split with a reverse index search so that your name can contain the #’s if need be. By default it looks for a single #, but you could replace this with ‘XXX123HACK‘ if that was the only unique string match you can manage. Make sure not to use the string value of ‘true‘.
- On my collector I like to filter by title. This is the $namevar. Sadly, this doesn’t support any fancier matching like in_array or startswith. I consider this a puppet deficiency. Hopefully someone will fix this to allow general puppet code here.
- Adding this to each resource is kind of annoying. It’s obviously a hack, but it’s the right thing to do for the time being IMHO.
Hope you had fun with this.
Happy hacking,
James
PS: [1] One side note, in the general case for custom resources, I actually think that by default duplicate parameters should be required, but that a resource could provide an optional function such as is_matched which would take as input the two parameter hash trees, and decide if they’re “functionally equivalent”. This would let an individual resource decide if it matters that you specified thing=>yes in one and thing=>true in the other. Functionally it matters that duplicate resources don’t have conflicting effects. I’m sure this would be particularly bug prone, and probably cause thrashing in some cases, which is why, by default the parameters should all match. </babble>
Fresh Stats Comparing Traditional IT and DevOps Oriented Productivity
This is a guest post by Krishnan Badrinarayanan (@bkrishz), ZeroTurnaround
The word “DevOps” has been thrown around quite a lot lately. Job boards are awash with requisitions for “DevOps Engineers” with varying descriptions. What is DevOps, really?
In order to better under what the fuss is all about, we surveyed 620 engineers to examine what they do to keep everything running like clockwork – from day-to-day activities, key processes, tools and challenges they face. The survey asked for feedback on how much time is spent improving infrastructure and setting up automation for repetitive tasks; how much time is typically spent fighting fires and communicating; and what it takes to keep the lights on. We then compared responses belonging to those from traditional IT and DevOps teams. Here are the results, in time spent each week carrying out key activities:
Conclusions we can draw from the results
DevOps oriented teams spend slightly more time automating tasks
Writing scripts and automating processes have been a part of the Ops playbook for decades now. The likes of shell scripts, Python and PERL, are often used to automate repetitive configuration tasks but with the newer tools like Chef and Puppet, Ops folk perform more sophisticated kinds of automation such as spinning up virtual machines and tailoring them to the app’s needs using Chef or Puppet recipes.
Both Traditional IT and DevOps oriented teams communicate actively
Respondents belonging to a DevOps oriented team spend 2 fewer hours communicating each week, possibly because DevOps fosters better collaboration and keeps Dev and Ops teams in sync with each other. However, Dev and Ops folk in Traditional IT teams spend over 7 hours each week communicating. This active dialogue helps them better understand challenges, set expectations and triage issues. How much of this communication can be deemed inefficient is subjective, but it is necessary to get both teams to onboard. Today, shared tooling, instant messaging, task managers and social tools also help bring everyone closer together in real-time.
DevOps oriented teams fight fires less frequently
A key tenet of the DevOps methodology is to embrace the possibility of failures, and be prepared for it. With alerts, continuous testing, monitoring and feedback loops that expose vulnerabilities and key metrics, teams are enabled to act quickly and proactively. Programmable infrastructure and automated deployments provide a quick recovery while minimizing user impact.
DevOps oriented teams spend less time on administrative support
This could be a result of better communication, higher level of automation and the availability of self-service tools and scripts for most support tasks. If there’s a high level of provisioning and automation, there’s no reason why admin support shouldn’t dwindle down to a very small time drain. It could also mean that members of DevOps oriented teams help themselves more often than expecting to be supported by the system administrator.
DevOps oriented teams work fewer days after-hours
We asked our survey takers how many days per week they work outside of normal business hours. Here’s what we learned:
| Days worked after hours | Traditional IT | DevOps Oriented |
| Average | 2.3 | 1.5 |
| Standard Deviation | 1.7 | 1.7 |
According to these results, DevOps team members lead a more balanced life, spend more time on automation and infrastructure improvement, spend less time fighting fires, and work less hours (especially outside of normal business hours).
DevOps-related initiatives came up on top in 2012 and 2013, according to our survey. There’s a strong need for agility to respond to ever-changing and expanding market needs. Software teams are under pressure to help meet them and the chart above validates its benefits.
Rosy Stats, but hard to adopt
How we got here
IT Organizational structures – typically Dev, QA, and Ops – have come to exist for a reason. The dev team focuses on innovating and creating apps. The QA team ensures that the app behaves as intended. The operations team keeps the infrastructure running – from the apps, network, servers, shared resources to third party services. Each team requires a special set of skills in order to deliver a superior experience in a timely manner.
The challenge
Today’s users increasingly rely on software and expect it to meet their constantly evolving needs 24/7, whether they’re at their desks or on their mobile devices. As a result, IT teams need to respond to change and release app updates quickly and efficiently without compromising on quality. Fail to do so, and they risk driving users to competitors or other alternatives.
However, releasing apps quickly comes with its own drawbacks. It strains functionally siloed teams and often results in software defects, delays and stress. Infrequent communication across teams further exacerbates the issue, leading to a snowball effect of finger-pointing and bad vibes.
Spurring cultural change
Both Dev and Ops teams bring a unique set of skills and experience to software development and delivery. DevOps is simply a culture that brings development and operations teams together so that through understanding each others’ perspectives and concerns, they can build and deliver resilient software products that are production ready, in a timely manner. DevOps is not NoOps. Nor is it akin to putting a Dev in Ops clothing. DevOps is synergistic, rather than cannibalistic.
DevOps is a journey
Instilling a DevOps oriented culture within your organization is not something that you embark on and chalk off as success at the end. Adopting DevOps takes discipline and initiative to bring development and operations teams together. Read up on how other organizations approach adopting DevOps as a culture and learn from their successes and failures. Put to practice what makes sense within your group. Develop a maturity model that can guide you through your journey.
The goal is to make sure that dev and ops are on the same page, working together on everything, toward a common goal: continuous delivery of working software without handoffs, hand-washing, or finger-pointing.
Support the community and the cause
Dev and Ops need to look introspectively to understand their strengths and challenges, and look for ways to contribute towards breaking down silos. Together, they should seek to educate each other, culturally evolve roles, relationships, incentives, and processes and put end user experience first.
The DevOps community is small but burgeoning, and it’s easy to find ways to get involved, like with the community-driven explosion of DevOpsDays conferences that occur around the world.
Set small goals to be awesome
Teams should collaborate to set achievable goals and milestones that can get them on the path to embracing a DevOps culture. Celebrate small successes and focus on continuous improvement. Before you know it, you will surely but gradually reap the benefits of bringing in a DevOps approach to application development and delivery.
Start here
For deeper insights into IT Ops and DevOps Productivity with a focus on people, methodologies and tools, download a 35-page report filled with stats and charts.
puppet matlab module
I don’t like repetitive work, and installing matlab counts doubly as so. Once I figured out the correct steps, I automated it with a puppet module. The downside is that the install takes a while because puppet needs to copy the iso locally. This is okay because I can be busy doing something else while this is happening.
Using the module is quite easy:
matlab::install { 'R2011a':
iso => 'puppet://files/matlab/MATHWORKS_R2011A.iso',
licensekey => '#####-#####-#####-#####', # provide your own here
licensefile => 'puppet:///files/matlab/license.lic', # get your own!
licenseagree => true, # setting this to true 'acknowledges' their (C)
prefix => '/usr/local',
}
You might notice that this supports installing multiple releases on the same machine. You will have to provide your own license key and license file.
This isn’t routinely tested, so if the latest matlab installer changes and this breaks, please let me know. Personally, I’d recommend you use octave instead, but if you really need matlab, hopefully this will make your sysadmin happier.
Enjoy a copy of the code:
https://github.com/purpleidea/puppet-matlab
Happy hacking,
James
PS: matlab also supports installs that use a licensing server. This module doesn’t support these at the moment, but you’re welcome to send me a patch. What a waste of hardware!
puppet rsnapshot module
Today I am releasing a puppet module for rsnapshot. The nice feature of this module, is that it lets you configure multiple different instances of rsnapshot, so that they could all run in parallel. Rsnapshot doesn’t support this directly, so this puppet module does the heavy lifting of separating out and managing each instance.
https://github.com/purpleidea/puppet-rsnapshot
The examples/ directory should give you a hint on how to use it. For everything else, have a look at the code, or feel free to leave me a comment. I hope you find it useful!
Happy hacking,
James
