Friday Jan 29, 2010

Farewell!

This is my last entry on this blog as I approach my final days at Sun. I'm not sure what my next steps are yet, but I'm hopeful. I am deeply interested in Email, mobile and collaboration technologies and I wish to continue working in this field.

I'm on Facebook, LinkedIn, etc (see "Social Networks" on sidebar), if you would like to contact me.

Thanks for reading my blog.

Wednesday Jun 17, 2009

Configuring CalDAV on iPhone 3.0 for Sun Calendar Server

The next release of Sun Calendar Server will be CalDAV-based. It is codenamed "Project Aries". We have a beta instance running internally, and with some tinkering, it can work on iPhone 3.0.

To configure for Sun's CalDAV Server:

  1. Go to Settings > Mail, Contacts, Calendars
  2. Tap "Add Account...", then "Other"
  3. Under Calendars, "Add CalDAV Account"
  4. Enter account information:
    • Server: [cal.domain.com]
    • User Name: [your username]
    • Password: [your password]
    • Description: [e.g aries]
  5. Tap "Next"

Here, if your CalDAV server is configured only on SSL, you might get a "Cannot Connect Using SSL" error. Just ignore and tap Save. Then edit the new CalDAV account, go into "Advanced Settings", change the port from 8443 to 443, and fix the Account URL to this format: https://cal.domain.com/davserver/dav/principals/[username]/ (don't forget the trailing slash).

Also see this entry by Arnaud on setting up against non-production CalDAV servers.

Configuring LDAP on iPhone 3.0

[Cannot Connect Using SSL]Another new feature in iPhone 3.0 software update is a built-in LDAP client. Previously, a third-party app such as Directory or LDAPeople is required.

To configure LDAP:

  1. Go to Settings > Mail, Contacts, Calendars
  2. Tap "Add Account...", then "Other"
  3. Under Contacts, "Add LDAP Account"
  4. Enter account information:
    • Server: ldap.company.com
    • User Name: [e.g. cn=First Last (employee ID), ou=people, dc=company,dc=com]
    • Password: [your password]
    • Description: [e.g. book]
  5. Tap "Next"

Unfortunately, it doesn't work with Sun's book.sun.com. It gives a "Cannot Connect Using SSL" error. A reader already reported that this could be due to lack of ca-cert three months ago so looks like this might be an open iPhone bug.

Come to think of it, this could be an underlying OS X bug. Using the Address Book app in OS X 10.5 configured for book.sun.com, if "Allow self-signed certificates" is checked, lookup works; if it is unchecked, lookup fails, even though book.sun.com has a legitimate chained cert with Versign as CA.

In the meantime, if you want to use LDAPS on iPhone 3.0, the workaround is to buy a third-party app.


[UPDATE Jun 23, 2009] Bug ID# 7000490 filed with Apple.

Configuring CalDAV on iPhone 3.0 for Google Calendar

Today, the iPhone 3.0 software update finally came out and about the first thing I did after upgrading mine was to check out the new CalDAV and LDAP functionalities.

This is how I configured CalDAV for Google Calendar:

  1. Go to Settings > Mail, Contacts, Calendars
  2. Tap "Add Account...", then "Other"
  3. Under Calendars, "Add CalDAV Account"
  4. Enter account information:
    • Server: www.google.com
    • User Name: [your username]
    • Password: [your password]
    • Description: [e.g. Personal]
  5. Tap "Next"

That's it! Now you can launch the Calendar app, tap "Calendars", then select the CalDAV calendar. Events can be created or modified on the iPhone and they'll be reflected on the server instantly.

Secondary calendars: This only shows the main calendar. To get your other calendars, you'll need to repeat the above for each (with distinct Descriptions otherwise it'll be confusing), then edit that CalDAV account, and under "Advanced Settings" change the "Account URL". You can get this Account URL by clicking the down arrow next to your calendar at calendar.google.com and selecting "Calendar Settings". You should find your Calendar ID next to the XML, iCal, HTML links in the Calendar Address section. Insert this Calendar ID into an URL like so: https://www.google.com/calendar/dav/ccogk69iprl3zyfa1ijeep0oj5@group.calendar.google.com/user . You can email this to yourself, then leverage the new copy&paste feature of 3.0 to avoid typos. Why is this so cumbersome? Here's a possible explanation.

Offline access: Works in both directions. Changes are sync'd upon network reconnection.

CalDAV or Google Sync? Either way will give you read/write access to your Google Calendar. A major deciding factor is whether you already have an active Exchange account (e.g. for work), since iPhone only allows one Exchange account at any time, and Google Sync uses Exchange ActiveSync protocol.

[Updated per suggestions in the comments]

Tuesday May 05, 2009

Blocking HTTP connections from an IP on Solaris 10

Today, I woke up to find that surl, one of the internal web apps I wrote, had stopped working and was returning "Can't open file: 'surl.MYI'. (errno: 145)", which means the database table was corrupted, a quick Google search revealed.

Fortunately, the repair procedure is simple enough:

mysql> REPAIR TABLE surl;
...
mysql> \^D

However, it didn't work. Turned out my / filesystem was full, so I deleted some files under /var/tmp, re-ran the REPAIR command, and surl was back in business.

Later, I began to investigate how / filled up all of a sudden; nothing particular stood out until I looked at the Apache access log, which shows that the web app was being hit by two IP addresses every 5 secs since April 9th! Unable to contact the owner of those systems, I decided to block connection from these IPs. It took a while to figure out the correct syntax to put in httpd.conf:

<VirtualHost 123.45.67.89>
...
<Directory /path/to/htdocs>
Order deny,allow
Deny from ww.xx.yy.zz
</Directory>
...
</VirtualHost>

It is critical to use the proper "Order" and not to put "Allow from all" in the final line as that would negate any previous Deny command. After, refresh Apache with this:

# svcadm refresh apache2

But that only resulted in Apache returning 403 response to every request. I want to block the requests completely, so I need to refuse these connections before they get to Apache. I considered using /etc/hosts.deny, but Apache doesn't support that. Then I found out about ipfilter in Solaris 10. I have used ipchains and ipfwadm on Linux many years ago, but I have no experience with ipfilter. After reading numerous online resources, it is apparent that ipfilter is a very powerful tool, but it wasn't very clear to me how to use it to do a very simple task, which is to block connection by IP. These are the steps that I took:

# svcadm enable ipfilter
# vi /etc/ipf/pfil.ap # and comment out your interface name, e.g. hme
# svcadm restart pfil
# echo "block in from ww.xx.yy.zz to any port = 80" | ipf -f -
# ifconfig hme0 unplumb
# ifconfig hme0 plumb <IP addr> netmask 255.255.254.0 broadcast <bcast addr>

The most tricky part is that the interface must be unplumbed and plumbed (or server must be rebooted which isn't an option for most I'd imagine). Until then, IP filtering is not active! I'm specifying the rule via STDIN instead of /etc/ipf/ipf.conf because I don't intend to make the block persistent across reboot, I'm hoping whoever is hitting my web server every 5 secs will notice and stop doing so.

These are also useful to know:

# ipf -Fa # flush all existing rules
# ipfstat -hio # show hits against all rules

I hope this will save someone else's time.

Tuesday Mar 17, 2009

CalDAV and LDAP coming to iPhone

Of all the new features Apple announced today in the upcoming iPhone 3.0 update, the two that get me most excited are CalDAV and LDAP support. It's great to see CalDAV come to mobile devices finally; I hope more smartphone manufacturers and developers will follow suit.

Many people wondered why Apple doesn't allow API access to the Calendar app. I had a suspicion it was because they had bigger plans for Calendar, and if they had allowed API access to Calendar data it would have precluded Apple from making these plans happen. Then came ActiveSync support, and now CalDAV. Perhaps Apple will open Calendar API access now?

Unfortunately, there isn't a lot of detail around CalDAV and LDAP yet, so we'll have to wait until summer to find out. I wonder how offline access to a CalDAV calendar will be handled? Will it be read-only, or will changes sync back to the server? Does LDAP integrate well with other iPhone apps like Mail and Contacts? Will it support LDAPS?


[UPDATE Jun 17, 2009] iPhone 3.0 was released today and I posted a few entries related to configuring CalDAV and LDAP:

Wednesday Dec 10, 2008

The Amazon sleigh

[Amazon sleigh] Did Amazon change their shopping cart in previous holiday seasons? Regardless, it's a nice touch.

In case you are wondering what was in my sleigh, it's a La Crosse Technology BC-900 AlphaPower Battery Charger.

Merry Christmas and Happy New Year to everyone.

Friday Aug 29, 2008

Managing Google Calendar from iCal

I use Google Calendar (or GCal as it's often called) at home to share upcoming events such as important dates, weekend plans, medical appointments, travel schedules, etc with my family and this is working quite well. After showing my mom how to use it, she has even eclipsed me to become the biggest calendar user in our family. We subscribe to each other's calendars so we can see who's busy or available at any time. We haven't gotten to the point where we invite each other over calendar though, which is good IMHO, because that's a bit too impersonal. :)

Since I want to see both work calendar and family calendar in one place, I opted to subscribe to them via iCalendar format. This gives me a read-only copy of the calendars in iCal; if I need to create or modify an event, I need to use the web interface. Ideally, I want to be able to manage both work and family calendars directly in iCal.

As if Google heard my wish, they added CalDAV support about a month ago. It was very easy to setup, and has worked fine for me except for a full-day outage about two weeks back. Here's what my iCal looked like before with iCalendar subscription (pic to the left), and here's what iCal looks like after with CalDAV enabled:

Before After

Now I can fully manage my family calendar from iCal. I'm one step closer to my ideal state! My biggest gripe with this setup is that to see my secondary calendars and subscribed calendars, I need to create a separate account for each. It would be much better if they just show up under one account. Is this due to a defect in Apple iCal, Google's implementation, or CalDAV protocol?

For my work calendar to become fully manageable in iCal, I have to wait a bit longer, as CalDAV support is coming to Sun Comm Suite Calendar Server in 2009.

[UPDATE] Here's a more complete list of UX issues with Google Calendar in iCal over CalDAV. Also, the developer of the Google Calendar Provider for Thunderbird/Lightning shared his thoughts.

[UPDATE 2009-06-19] Secondary calendars are indeed accessible if you enable them under the Delegation tab in Preferences > Accounts.

Monday Aug 25, 2008

Review of CEAS 2008

Last week, I attended CEAS 2008, an Email and anti-spam conference where researchers from all over the world presented latest techniques they have devised against spam. Most of them were pretty heavy on statistical analysis. I have to admit I haven't seen this many math formulas since college, or heard terms such as "OSBF-Lua", "(RO)SVM", or "TREC" before from any anti-spam product vendors.

There were two sessions that I found to be particularly interesting: A Survey of Modern Spam Tools by Henry Stern of IronPort/Cisco and Fighting Spam: Gmail's Story by Brad Taylor of Google. Henry talked about Dark Mailer, Send Safe and Reactor Mailer—the last one is responsible for 40-50% of all Email traffic on the Internet—and showed us an example of how spammers could use the 'rndline' template macro to generate 28.5 quadrillion unique messages. Brad talked about some of the anti-spam measures Gmail takes; he couldn't share all the details for fear that someone might try to game the system with that knowledge. The Gmail "Spam Czar" is no doubt a celebrity in this circle, but he wasn't the only one; Eric Allman (who developed sendmail) and David Crocker (author of RFC 822) were among those in the audience.

Social Honeypots: Making Friends With A Spammer Near You by Steve Webb of Georgia Tech was pretty entertaining as well, for it's evident that spammers are reaching out beyond Email to social networks, but according to Steve's test on MySpace they seem to only target male users.

Perhaps the best part was meeting more than a handful of smart people genuinely interested in messaging. I'll definitely try to come back next year, especially if there's more focus on those other aspects of anti-spam, like maybe a best practice talk on DKIM or SPF, how to defeat SpamAssassin (or another anti-spam tool), how to avoid being blacklisted, etc.

Monday Jul 28, 2008

iPhone 2.0 firmware and IMAP IDLE

The answer is still no, firmware 2.0 on the iPhone does not appear to support IMAP IDLE. The push email feature works only with Microsoft Exchange, MobileMe, and Yahoo!. If you go to Settings > Fetch New Data > Advanced, and select an IMAP server, only two choices are given: Fetch [periodically] or Manual. No Push. This is reaffirmed by my IMAP telemetry log. Just like in firmware 1.x, iPhone's mail client never sends the IDLE command.

Since I installed the new firmware on my original iPhone, I've noticed drastic battery life reduction. Following Apple's advice, I turned off push email and saw significant improvement. With push email enabled, I had to recharge it almost daily; now my iPhone can go 2-3 days between charges, which is still more frequent than, but closer to my experience on 1.x firmware.

Wednesday Apr 23, 2008

Is IMAP IDLE the right solution

To facilitate push Email to mobile devices? I'm doubtful, here are the reasons:

First, IMAP IDLE relies on keeping the IMAP connection alive. Mobile devices are more likely than desktop clients to experience network errors and timeouts as they travel. And devices capable of both Wi-Fi and cellular, like Apple's iPhone, automatically switches from one network to the other, terminating the connection as they do so. I feel that a true push solution should not depend on a persistent connection.

Second, an IMAP IDLE session only monitors a single mailbox. If you need to watch multiple folders simultaneously (like if you use server-side filtering or subaddressing), you need multiple sessions. Is any client capable of that?

Third, maintaining the session not only drains the battery, of which mobile devices have limited amount, it also hogs the data channels so I don't think telcos like that idea very much.

Michael told me OMA-EMN uses sideband SMS for notification, so it's designed to be friendlier to battery and network operators compare to IMAP IDLE. But I've yet to come across a client that supports OMA-EMN.

Tuesday Apr 22, 2008

Race between spammers and anti-spammers

In recent months, several companies have released their statistics and outlook on spam.

  • Postini blocked 47 billion spam messages, over 320 Terabytes of spam in October 2007 alone.
  • Barracuda Networks reports that up to 95% of Emails are spam in 2007.
  • Sophos suggests that 92.3% of email sent during the first quarter of 2008 was spam.
  • Symantec Brightmail calls Europe the new King of Spam, taking the title from North America for the third month in a row.
  • Google says incoming spam to Gmail is declining. Some disagree.

The race between spammers and anti-spammers has been going on for years, and spammers have the upper hand now and for the foreseeable future because their opponents are constantly playing defense.

As amazing as computers are at pattern recognition, randomizing a pattern is still magnitudes faster and cheaper than to resolve one. Take CAPTCHA for example. Or consider the number of turns it takes to scramble a Rubik's cube and to unscramble it.

The problem with current spam detection techniques is that they are largely built on pattern recognition, be it phrase filtering, Bayesian, fingerprinting, DNSBL, heuristic analysis or profiling. As soon as an anti-spam algorithm finds a reliable way of countering a certain kind of spam, all it takes for spammers to defeat that and to launch a new wave is to randomize their attack further, and it's back to playing catch up for the blockers. With the vast amount of resources these spammers have, their operation can only become more and more efficient.

Anti-spammers can't win by this strategy, no matter how good they become at it. If they want to win the race, they must start playing offensive, and by that I don't mean legal actions, because spammers are not regulated by any country.

Tuesday Mar 25, 2008

Haven't they heard of phishing?

I received an Email from my health insurance provider. A few days before, I had used the "Contact Us" section of their website to send them a question about my account.

At a quick glance, the Email seems unremarkable. But instead of a response to my question, it contains instructions to decrypt my message, which involves opening the HTML attachment as a first step (red flag #1). Normally, I don't open any HTML attachment from unknown sources, to avoid validating my address to spammers, but this one should be fine, right? I look at the From: header, it looks legitimate, but of course it could have been forged (red flag #2). Then I expand to full header view, and the set of Received headers supports the From: identity. The earlier Received headers could have been doctored, but I trust the one added by Sun's MX gateway. After all, I've been anticipating an Email from them, so I decide it's ok to proceed.

When I click on the attachment, Mail.app prompts Are you sure you want to open the application "securedoc.html"? Could this be a trojan horse disguised as an HTML file? I examine the source of the Email, and find that the MIME Content-Type field of the HTML attachment was mislabeled as "application/octet-stream" rather than "text/html" (red flag #3), so there's no foul play here. I drag the HTML icon over to Safari and it opens.

According to the instructions provided, once I open the HTML file I should get two choices. Either click on the "open" button and an applet will be downloaded and installed to my C: drive, or click on the "here" link and my browser will open the secure email without any applet downloading. Since I'm on a Mac, I don't have a C: drive, and I prefer (just in case this indeed is a trojan horse) not to install any applet, I look for the "here" link. But there is none (red flag #4). Left with no other choice, I click on the "open" button.

Fortunately, this correctly initiates the process of key retrieval from the insurance provider and message decryption. I'm glad it didn't ask for my membership number and password, which would've raised another red flag or two. Below the response, an underlined heading says "Original Message Excluded" (red flag #5). The footer/signature has only three words, "PostX Secured Email," which is the technology behind all this.

Although the response provides the information I asked for, so many red flags went up in my head, I can't help but wonder if they had run this process by a messaging consultant. Regardless, here's some free advice:

  • Solution to #1: Don't ask or expect your recipients to open any attachment or URL, until you have built confidence with them.
  • Solution to #2: Sign your messages with a certificate to prove your identity.
  • Solution to #3: Label your MIME fields correctly, it's unprofessional not to do so.
  • Solution to #4: Make sure you provide instruction that matches actual experience precisely.
  • Solution to #5: Include the original message (where applicable), ideally before you ask for user action.

Come to think of it, these suggestions can help any Email marketer wanting to distinguish themselves from phishers/spammers.

Monday Mar 24, 2008

Reduce The Fine Manual, please

Last Friday, we got a Samsung Blackjack II (not by choice) in the family and naturally, being the Chief Geek, I had the "honor" of setting the phone up. I have to say this Q/BlackBerry killer turns out to be quite a disappointment. The WM6 user interface is convoluted, the device is cluttered with AT&T junkware and the thumb keyboard feels cramped. Motorola's mobile enterprise messaging and calendaring suite called Good is a pain to use.

But the biggest turn-off is the booklet of manual that comes with the phone. Why couldn't the product designers make a feature so intuitive that it needs little to no explanation?

As WSJ's review of the Flip says: "[Devices should be] so simple, mastery is immediate, and so is your sense of pride and happiness."

The importance of that latter part is what the collective Microsoft-AT&T-Samsung-Motorola team did not understand and embrace. While they packed the smartphone with an impressive list of functionalities, they also overlooked the experience of unboxing the gadget for the first time. Result: customers are overwhelmed with features they can't use unless they read the manual first.

In contrast, Apple's approach to product design reflects in their minimalistic user manual. The entire iPhone manual comes on a single double-sided sheet with large screenshots.

If you make a consumer product, it's best to assume they won't read the manual at all.

If your product has features that are fragile or unusable without some manual reading up front, you need to rethink your design.

If you're improving on a product, slashing the existing manual by 50% or more should be a top priority. Then slash it again. Why do it if people aren't going to read it? Because the exercise forces you to create intuitive features, leading to a better experience, therefore a better product.

Thursday Feb 21, 2008

iPhone and IMAP IDLE (part 2)

This is a follow-up and correction to my previous post: iPhone supports IMAP IDLE.

After more testing, turns out iPhone does not support IMAP IDLE (on a Wi-Fi connection, firmware 1.1.3), it instead Auto-Checks every 2-5 minutes despite the Setting says don't. I verified this by enabling IMAP telemetry in Dovecot 1.0 through its rawlog facility. Even though the server advertises IDLE in its CAPABILITY response, iPhone's client never issues the IDLE command.

Is that a loss? Probably not. According to this thread, beta testers of the iMapIdle utility report significant battery drain while IDLE runs on a Wi-Fi connection. On an EDGE connection, the drain is not as bad. But when iPhone switches between Wi-Fi and EDGE, the IDLE session gets dropped. For these reasons, I can see why Apple designers chose to not implement IMAP IDLE on iPhone.

About

I currently live in San Francisco Bay Area. For the past seven years, I have been designing and building messaging solutions for Sun.

Search

Categories
  • General
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today