Sunday, 28 May 2017

Avoiding Mysterious "Permission denied" Errors During Windows Development: Check Your Services!

A short post to clearly answer a question I've seen un-answered, or answered incorrectly on at least three other major programming forums: if you can compile your program the 'first time' -- in Eclipse, mingw, golang, whatever -- and run it, but the next build/run cycle you get "Permission denied" from the linker or 'go build', or similar -- check that you don't have the "Application Experience" service disabled. It must be set to Manual, Automatic, or Automated (delayed start)... just not Disabled. Otherwise you'll experience this issue, regardless of language, compiler, or IDE, at least on Windows 7.

If you're like me you are in the habit of turning off all sorts of Windows Services on your personal machines to minimize the background crap going on and reduce system memory usage. Well, it's always a matter of tuning to determine what's really required and what isn't. For programming with rapid edit/build/run/debug cycles, it's common to run into this situation.

I have no idea (nor does anyone else, it would seem --Microsoft isn't telling on its own forums) exactly why this service must be enabled to prevent the OS from locking a recently-terminated program executable on disk.

Perhaps (and this is a wild-ass guess), due to the way Application Experience wants to monitor programs, and report if they crash, there's some other OS component in Windows that is waiting for a handshake from the Application Experience service before letting the exeuctable be removed. This might make sense if one needs to send dumps of the .exe or something as part of a crash report.

Anyways... the info on the 'net is confusing about this, so I decided to note it here mostly for myself, for posterity as I've run into it at least 3 times on various Windows systems doing development and it's highly annoying.

Thursday, 26 November 2015

ZINK hAppy Printer Setup on Windows 7

ZINK Imaging Inc. makes some neat devices, the hAppy and hAppy+, which are basically super-fancy labelmakers. From what I understand the company was founded by refugees from Kodak after they shut down the Polaroid division, just after they had developed a cool new digital version of the Polaroid process.

I picked up a hAppy printer on eBay a while ago, thinking it might be neat as an alternative way of producing my c@rd password gen/recall cards. The fact the device can print directly onto shiny peel 'n stick labels, in multiple widths, is quite nifty. The widest roll is 2" (50mm) which just so happens to be nearly the width of a credit card.

While it's marketed at the craft market (think soccer moms and kids' birthday parties), it could be put to use in so many other ways. The manufacturer's site and installation instructions are focused completely on its AirPrint capabilities: printing from a smartphone via iOS or Android over WiFi, using their proprietary apps (which, to be honest, are pretty good for the craft market -- easy to use and lots of neat clip art to get designs done quick for those birthday parties, weddings and craft parties).

However, I want to use it to print my own stuff, from a laptop at a mobile kiosk, using my own algorithmically-generated security cards.. I can do it from the smartphone but it requires a lot of manual fiddling with resizing/cropping in their phone app, even though my c@rd generator app spits out fully-specified, sized images ready for printing. I even encode the exact print size in the EXIF data so it should really be 100% automated.

The only issue was that ZINK doesn't officially ship or publish *any* Windows USB drivers! The hAppy printer has a microUSB port, so I wondered if it could just act as a regular printer.

Turns out, the answer is YES. ZINK didn't help one iota, though. Multiple emails to their tech support just went into a black hole. They've either already moved onto the next product line, or just have no real humans watching their emails. Perhaps they're already going bankrupt, who knows. Bad customer support anyways.

Lots of info on the 'net about using the ZINK range with Apple's AirPrint is out there; which leads one to believe that the ugly Apple iTunes software (which for some gawd-awful reason is the only official way to obtain AirPrint capability on Windows!?) is the path to printing bliss here.. but it is NOT the right solution. Don't put that crap on your machine, it's a dead end for this device; I managed to get my Windows 7 PC to 'see' the hAppy via Wifi, but it wouldn't print anything, just giving obscure errors in the print spool manager.

So... this device shows up in Windows 7 (64-bit) upon initial plug-in via USB as a generic 'Unspecified' entry in Control Panel\All Control Panel Items\Devices and Printers. You'll see a blank box labelled just 'hAppy Printer', in the 'Unspecified' Section below 'Printers and Faxes'. That means it's showing up as a USB device, but Windows doesn't yet know how to print to it.

Right-clicking into Properties, you'll see the usual device Properties dialog, with the usual buttons -- except that 'Update Driver', 'Disable', 'Uninstall', etc. are greyed out. The only one available is 'Driver Details', so click that and you'll also see it says no drivers have been installed for the device. Not promising.

However, Windows has a little-known class of support drivers, 'USB Printing', which sit beneath more specific vendor printer drivers. And, it turns out, ZINK does in fact have a Windows USB driver for this printer! It's just not published anywhere outside of Microsoft's Windows Driver database. So... the only way to get this installed is to go through the Control Panel:

1. In Control Panel -> Devices and Printers, choose 'Add a Printer'.

2. Choose 'Local Printer', and change the dropdown beside 'Use and Existing Port' from LPT1: to USB00x: (whichever one is there; if you have an existing printer it might be USB002: or USB003: or higher.. )

3. Click Next. If the next dialog doesn't have a manufacturer named 'ZINK' in the left pane, you need to click 'Windows Update' and wait about 5 minutes (or longer!). It eventually will come back, and ZINK will be hopefully new in the left pane's list of manufacturers.

4. Choose 'hAppy' from the printer list on the right pane. DO NOT choose 'hAppy XPS', (not that I've tried it, but XPS is a virtual printer driver, which we don't want -- unless you want to print to files for later physical printing.)
5. Complete the wizard and now in the Control Panel -> Devices and Printers, you should now see your new 'ZINK hAppy' in the Printers and Faxes Section.

Now, printing to labels using Windows Explorer's default right-click -> Print will work. You'll have to fiddle with setting default Printer Preferences, setting a Custom paper size preset to match the width of your roll, and so on (I'll write details of that later if anyone cares -- it's a bit fidgety, but relatively straightforward). Be mindful of the print dialog's 'Scale to Fit' settings etc. which will affect the printout size. With two or three tries I was able to come up with presets that printed out at my expected size. The 'Kiss Cut' vs. 'Full Cut' setting didn't seem to make any difference, however -- the hAppy always auto-chopped my print after it was done, which was a bit annoying, but no biggie.

Strange that ZINK hasn't officially published that this driver is available; it seems to be relatively complete, enough so that they submitted it to Microsoft's official driver database. It seems aware of all of the device's settings (normal vs. Vivid print mode, roll widths, etc.). It even reports the remaining roll length, so it seems to be the real deal.

No idea if this is installable, but here's the driver folder from my Windows 7 x64 System. As Admin, extract to C:\Windows\System32\DriverStore\FileRepository.

UPDATE: I've been told some people have trouble downloading the above archive from my Google Drive. Alternate Download Link

Friday, 18 July 2014

Google Yanks Yet Another Useful Feature with No Warning: Labs "Add any gadget by URL"

Boo Google, boo.

Well, if anyone needed another example of why it's just a bad idea to rely on Google APIs for anything important, here's another one. Today I saw a teensy note appear at the top of my gmail announcing the Labs "Add any gadget by URL" gadget is being deprecated.

I used this feature to developer my "Hashcash mint" gadget, and it's the easiest way to add such gadgets to the sidebar in one's gmail.

I've sent them feedback asking what migration strategy one should use and how exactly someone is supposed to embed gadgets in their gmail page now.. I hopefully will hear back within the next few days.

Perhaps I need to look further into the OAuth2 system or something; but I fear the response will be something like "Port it to a Chrome extension" (which I've already done, but that's not cross-browser is it?).
Anyone else know of APIs which allow embedding of one's custom gadgets into pages like GMail, Calendar etc. without using some kind of Google walled-garden to host them?


EDIT: No response yet.. the entire Google Developers section on gadgets is woefully out-of-date. I submitted about 4 support requests relating to dead links, missing sections, and just plain wrong info. Sigh. I get the feeling the entire gadget API has been left out to die, at least for people outside of Google.

Well if so, I'm glad I've only written one Google gadget and chrome extension, rather than investing more effort. Google just drives away devs by constantly deprecating things willy-nilly.

Saturday, 11 January 2014

mt-crypt - a simple stream cipher based on the Mersenne Twister PRNG

It's often said: "Anyone can write a crypto algorithm they can't break themselves". True enough, but that doesn't mean it should be forbidden to study or write crypto software because, well, one isn't an expert in studying or writing crypto software -- that would be a Catch-22 of sorts.

What's the entry point then, for someone who wants to understand? One learns by doing.

In light of the Snowden leaks of 2013 I think it's time that writing crypto became something to be encouraged rather than discouraged. Treating crypto software as a 'read-only' institution can't work -- we simply cannot trust a few blessed experts to write our crypto libraries and utilities for us. People outside of the shadowed halls of three-letter agencies must gain their own expertise, or at least become familiar enough with the concepts to ask intelligent questions, look for backdoors and weaknesses, try to apply new defenses to what should be private data, and just generally not be passive victims.

It has become more important than ever that there be a diversity of implementation, and that programmers who might have been hesitant before to write crypto programs start doing so.

I'm not saying one should apply any ol' system to important data. Algorithms must be carefully examined by many eyes to look for problems and determine the real security of any new system. But if everyone's afraid to write new systems, there won't be very much to examine, and not enough alternatives to keep those who would spy on us off-balance.

If the NSA's goal was/is to subvert the most prevalent cryptosystems, it makes sense to ensure there are no prevalent cryptosystems. We need to take the side of the mythical Hydra, whose winning strategy is to grow two heads for each one cut off, making things even harder for the attacker.

To do this, we all need to better understand just what strong crypto is and how to implement it so there are more choices for everyone to use, and no 'master key' -- no 'most popular solution' -- for those that would spy indiscriminately on us all by exploiting a single weakness. If the haystack cannot adequately hide the needle, we need to throw on a lot more hay, and add a few million more needles in there. If the opponent's resources are superior, an open battlefield no longer is tenable and we must use guerilla tactics -- divide and conquer, stay in small groups, change routines constantly.

[OK I'm done with the metaphors... and I'm probably on all the watch lists now :/]

With the above in mind, I started reading about alternative, non-NIST algorithms to see what's out there and how I might implement my own crypto utilities. I started out trying to implement a standard block cipher along the lines of DES or AES, trying to understand what S-boxes (substitution) and P-boxes (permutation/confusion) did, and how to use them.

Most block ciphers use multiple rounds of (S-box, P-box) manipulations, one followed by the other applied in multiple rounds with some sort of key schedule, to achieve an invertible transformation. Lots of arcane math goes into S-box design which made me a bit queasy.. I still need to learn a lot about that before I'm confident about designing such a thing.

Another big worry with block ciphers is the problem of padding. If you're trying to use a block cipher on files, or when encapsulating packets, the file as a whole, or each packet's payload, may not be a multiple of the blocksize; so the ciphertext must be padded somehow to be a multiple of the blocksize. There are 'padding oracle' attacks that make it dangerous to use block ciphers unless one understands the block padding issues completely. See here or here for some good explanations on how one can 'unzip' an encrypted CBC-mode block train if the padding scheme is known. Indeed, having any predetermined format or structure outside of the ciphertext itself can open things up to oracle attacks it seems.

So a stream cipher seems to be a safer entry point for learning and research for a would-be crypto programmer. A stream cipher doesn't encrypt a block at a time, rather it's a byte- or bit-oriented system that can handle arbitrary-length data. There is no concept of a blocksize, and thus no opportunity or temptation to introduce 'meta-data'. Usually this requires a 'cryptographically-secure PRNG' (pseudo-random number generator); that is, one that is very hard to predict so long as the seed (which is mathematically derived from the key) isn't known.

A cryptographic PRNG must meet a much higher standard for randomness and non-predictability than the 'usual' PRNGs used in standard libraries and games...

I knew of the Mersenne Twister (MT), a really good PRNG with an astoundingly huge period (2^19937-1). That sounded like a good candidate, but the literature says even that isn't cryptographically-secure enough, since despite a long period it still is subject to prediction attacks given enough PRNG output. The authors of MT wrote a paper addressing this, with an ingenious and simple scheme to harden the use of MT -- this consists of two things: a) a non-linear transformation on the PRNG output, and b) throwing away most bits of the PRNG output before combining with the plaintext. It would seem this makes it very hard indeed to know the state of the PRNG and to predict its output as used with a stream cipher.

A distinguisher is still plausible (see here), but no one has published a full-on key recovery attack as of yet and the researchers make no assertion that this is even possible from the distinguisher. The so-far-theoretical attack lets one determine that a stream of ciphertext likely is using Mersenne Twister as its underlying PRNG by observing 2^50 bits of contiguous ciphertext.

Now that's still a lot of data -- 2^47 bytes. Though some claim this cipher is 'broken' in the pure cryptographic sense, it seems it would be perfectly salvageable if one were to re-seed conservatively.

The idea would essentially be to re-seed on some interval between every 2^8 and 2^24 bits (2^16 to 2^32 bytes), based on the PRNG stream itself; thus a sort of (perfect-?)forward secrecy on the reseed schedule would be maintained (an attacker would need to first predict the MT output itself to know when the next re-seed would occur; but they shouldn't be able to do that, since the re-seeding would always occur at much less than 2^50 output bits).

The simplicity of the cryptMT stream cipher seems compelling unless someone proves otherwise and the authors' follow-up algorithm, crypMTv3, is much more complex from a design and code perspective (they focused on speed-ups for SIMD instruction sets, which is nice but I'd rather have a simpler algorithm). Crypto designers want security, but as we've learned from the recent Heartbleed fiasco, we also want simplicity of design. Complexity introduces the very real possibility of flaws that undo all other efforts.

[Note to self: the attack described in the above paper talks about LSBs of the 8 MSBs of the accum output. Perhaps some kind of xor-with-parity of the internal cryptMT accum value could further obfuscate the LSB to harden against this attack, or using multiple MT generators with divergent seeds and states, combining them via XOR, could make it infeasible to predict a single MT stream state?]

Wednesday, 27 November 2013

Gmail Hashcash Notifications To Your Phone

Gmail Inbox Setup for Hashcash Notifications To Your Phone

If you install my Hashcash for Gmail google script to scan & validate your incoming email according to the presence or absence of hashcash stamps, and you just want notifications for verified emails coming in, you can tweak how the inbox notifies you on your mobile phone. Here's one possible setup:

It's quickest to set this up from the desktop:

  1. From the gmail web interface Gear->Settings
  2. Inbox tab
  3. Inbox type 'Priority'
  4. For Inbox sections 1-4, Options->Remove Section
  5. Redefine Inbox section 1. Options->More Options...->Show All From Label [#$]
  6. Save Changes

Then on your Android phone, configure Gmail:
  1. Menu->More->Settings
  2. Account Settings->[your email address]
  3. -Check Priority Inbox (make default for this account)
  4. -Labels to Notify
  5.   ->Inbox: off
  6.   ->Priority Inbox: off (was: [subtle, always, notify for every msg])
  7.   -> [#$]: [subtle, always, notify for every msg]
NOTE: This is a pretty strict set-up in the respect that you won't get notifications for anything that doesn't have a valid Hashcash stamp. Not too useful until most people in your local network also are using Hashcash. That's the problem with network effects.. the usefulness of a thing only becomes apparent once many people are using it together.

If you look into Gmail's Priority inbox rules it's a pretty powerful way to tailor how/when you get notifications.

Monday, 18 November 2013

Tugging at the chains 'twixt myself and The Cloud

[Otherwise, less poetically-entitled: Screw you, NSAGoogle. I'll make my own damn webmail, with blackjack and hookers. Thing is, I actually -want- the webmail too. Really.]

2013-11-18: experiments setting up my own webmail server with IMAP and (eventually) OSS webmail client access

Part I: The MTA and IMAP service (dovecot), plus Thunderbird client access [DONE]
Part II: Webmail client for easy access anywhere (TODO)
Part III: End-to-end encryption using clients in (I) and (II), for anyone (TODO.. and non-trivial, I know)

1. Exim4:
   * just install using debian defaults
   * edit /etc/exim4/update-exim4.conf.conf for your smarthost
     (a smarthost is an SMTP server that will relay mail for you; usually
      smtp.<your_isp> -- note many ISPs these days don't allow any outgoing email
      even from their own customers' IP blocks. Complain bitterly to them that they're
      breaking the internet, not that it will do any good... other than some cathartic anger on your part. Then go find another provider, if you can.)

2. Get an IMAP server:
  apt-get install dovecot-imapd dovecot-sqlite dovecot-pop3d

   * Nice! Top of /etc/dovecot/dovecot.conf:
     "If you're in a hurry, see"

2.1. Configuring dovecot
  2.1.1 Authentication
   * [Further research here is required. I ended up using the default 'passwd-file'
      accounts rather than 'passwd' account auth which would use PAM to match against
      the system user accounts. It's safer to use 'passwd-file' anyways since that
      means users' email accounts don't have to use their shell accounts.]

  2.1.2 Mailbox setup/privileges
   * My deb system appears to use 'mail' group for /var/mail, so I set
     mail_privileged_group = mail,
     #! mbox_very_dirty_syncs = yes
     #! maildir_very_dirty_syncs = yes
     .. in /etc/dovecot/conf.d/10-mail.conf

  2.1.3 IMAP Compatibility Options
   * Since I like Thunderbird, enable the following IMAP workarounds
     in 20-imap.conf:
     imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags

  2.1.4 SSL setup
   * Consider after getting things going, enabling SSL. Generate a self-signed cert
     using Dovecot's doc/ script (see _SSL_ link)
   ** NOTE SSL is required by default, you have to set disabled_plaintext_auth = no

    i) Go to
    ii) Download the two files doc/ (or find in the downloaded dist files),
        and doc/dovecot-openssl.cnf
    iii) Edit dovecot-openssl.cnf to match your site install; also edit
         itself if you want to lengthen/shorten cert expiry and/or the name of the
         generated cert (default: /etc/ssl/private/dovecot.pem)

  2.1.5 IMAP account (server-side): see
    Default appears to be 'passwd-file' which sets up a separate username/pass DB for
    IMAP access. One could set up 'driver = passwd' to use the server's local user
    accounts, but that would also grant shell access (usually). Probably a good idea
    if setting up a public server to use the default.

    i) edit /etc/dovecot/conf.d/auth-passwdfile.conf.ext
       Set scheme=SHA512-CRYPT
    ii) $ doveadm pw -s SHA512-CRYPT -u <username>
    iii) cut and paste the resulting string into /etc/dovecot/users
    iv) edit /etc/dovecot/conf.d/10-auth.conf
        auth_mechanisms = plain login  (? Not required apparently)
        (default install only has 'plain')

3. Get a desktop or web-based mail client. I went through a few here:
  x IlohaMail - obsolete, the whole domain is gone, even though it's still a package in
    Debian repos and written in PHP4. So, no docs. Doesn't support POP-over-SSL or other
    encrypted auth schemes due to its age; too bad, it was easy to install. -REJECTED-
  x <lots of other webmail clients, too complicated to get going> -REJECTED-
  x <RoundCube? Still need to evaluate>
  x <MailPile? Looks promising, actively developed but very incomplete. They are working
     on integrated PGP/GPG support, perhaps a post-Snowden emphasis on e2e-crypto? Hope!>
  x .. or a desktop mail client such as Thunderbird.
      Thunderbird Account Settings:
        Server Settings: Server Type: IMAP Mail Server
                         Server Name: <server hostname>
                         User Name: <username> (without @domain.tld)
                         Security Settings: Connection security: SSL/TLS
                         Authentication method: Normal password (?)
        Outgoing Server (SMTP): Description: (your choice)
                                Server Name: <smtp smarthost server hostname>
                                Port: 25 (** NOTE default is 587, won't work!)
        Sec. and Authentication:
                                Connection security: None
                                Authentication method: No authentication

Monday, 23 September 2013

Hello World Blinky for ezSBC2 LPC1347 in Assembler

Doing things the hard way sometimes is the best way to learn. I've been meaning to learn some embedded ARM-fu for years, and just never got around to it since AVRs are just so darn easy (or familiar, in my case).

I like using a simple text editor and command-line toolchain but since most ARM vendors have wrapped up all the details of their chip startup in wizards within their huge IDEs, it's a pain to figure out just what's happening and what is required to bootstrap a given chip.

For my experiment, I chose the LPC1347 microcontroller from NXP, on a simple board called the ezSBC2. This chip has a neat built-in ROM that makes the chip virtually unbrickable -- pull a certain pin on power-up and the chip appears as a USB flash drive, and you just paste in a new 'firmware.bin' file with your code, reset and it's reprogrammed. Couldn't be simpler.

However, it took me two full nights' worth of searching online to find any kind of 'Hello World' reset startup code for the LPC134x chips. The best I could find was some for the related LPC1766, which was close enough for me to eventually figure it out. See below for download. I had to modify the startup.S, main.S and linker script to fit the LPC1347 memory map and peripheral registers.

One thing that isn't spelled out anywhere is that the Cortex-M chips are very different from traditional ARM chips in a lot of ways; for instance, the vector table isn't composed of branch instructions -- it's just a list of absolute addresses, like the old 68000 (and at vector 0x0, just like the 68000, is the initial system stack pointer value). They also don't have nearly as many contexts as traditional ARM (IRQ, FIQ, etc.). I'm still reading up on all the differences but it makes bootstrap code for 'standard' ARM totally inapplicable.

As for NXP-specific differences: not mentioned explicitly on ezSBC's website, but buried in the lpc1347 user manual, is the fact that one must put a 32-bit checksum in the first reserved vector 0x1C to validate the code. Otherwise the USB programmer won't accept the firmware.bin file. I've included a quick and dirty C program in the ZIP file below to patch up the binary after linking.

Hello World Blinky in Assembler for ARM Cortex-M3 NXP lpc1347

ezSBC2 ARM board

NXP lpc13xx User Manual