Build Visibility In

Visibility has been a constant theme for this blog. Elsewhere I've used the phrase build visibility in to emphasize the need to integrate visbility requirements into the build and design phases of any technology project. Visibility should not be left as an afterthought. Building security in is required as well, but how can you determine how security is working if you have no visibility?

Based on my experiences with technology deployments since the late 1990s, I've realized that the following cycle defines just about every project I've ever seen.

The cycle is Feature -> Management -> "Security" -> Visibility.



I am seeing this cycle at work in the mobile device space right now. Hardly anyone is thinking about how to determine if a mobile device (Blackberry, etc.) is compromised. The best we can do is imagine the sorts of attacks that might be happening to our mobile infrastructure, without visibility regarding how those devices might already be under attack.

I call this operating only within the Decide -> Act part of the OODA loop (Observe -> Orient -> Decide -> Act). We do it all the time in digital security. I called it Soccer Goal Security in 2005.

Does this cycle resonate with anyone?

Read More

Question on NSM Scaling

A long-time TaoSecurity Blog reader sent me the following question:

I have a question about scaling NSM in regards to large, complex enterprises that transmit countless gigabytes of data per day.

Last month I interviewed for a position with a large wireless company and the hiring manager was familiar with your work, so as I attempted to extol the value of NSM and explain how I thought that NSM could benefit this organization, I was told by the hiring manager that he felt that NSM worked with small organizations, but did not scale well with organizations of a certain size.

I am curious if you have ever had to counter this type of argument and how you addressed it.

This is a common question. I'll need to address it concisely and precisely in an updated edition of Tao. A few recent posts come to mind, like Requirements for Defensible Network Architecture: Monitored, NSM vs Encrypted Traffic, Plus Virtualization, and Network Security Monitoring Lives. A few principles come to mind.

• Concentrate on infrastructure you own, not necessarily infrastructure you support. In other words, I don't advocate full NSM for ISPs watching customer links. That may be the issue mentioned in the question, i.e., a wireless company might think NSM is inappropriate for watching customer traffic. I would probably agree with that.


• Monitor at trust boundaries. The places where the infrastructure you own touches infrastructure you do not own is likely to be the place where you need additional visibility.


• Monitor what you can, given your technical, political, and legal constraints. You may not be able to continuously capture full content data on 10 Gbps links with commodity hardware, or even specialized hardware. If that is too expensive, then don't do it. However, deploy the capability to capture at those locations when necessary. Better to be prepared than to struggle with workarounds or emergency deployments in a crisis.


• Solutions can be engineered for almost any environment. I guarantee organizations larger than those in the question are already doing intense monitoring. If you don't believe me, look at the history of wiretapping during the last administration. Outside of that case, organizations like mine are deploying hundreds of sensors around the world. NSM can scale if you engineer it properly and hire people who know what they are doing!


• Don't make NSM a hammer and every security problem a nail. NSM is one way to gain visibility and situational awareness. It may be worthless to deploy sensors doing traditional NSM on a link that only sees SSL-encrypted traffic between two point systems, or between the Internet and a SSL-only system. In cases like that, the first option might be to deploy host-centric monitoring and logging on the asset in question.

Thank you for questions like these -- please keep sending them. They make good sections for a new book.

Read More

Thoughts on Security Careers

Several recent blog posts have discussed security careers. I'll start with Anton Chuvakin's post A Myth of an Expert Generalist:

Lately I’ve run into too many people who [claim to] “know security” or are [claim to be] “security experts.” Now, as some of you recall, I used to do theoretical particle physics before I came to information security. In my physics days, I’d be pretty shocked if I were to meet a colleague in the hallways of the C.N. Yang Institute for Theoretical Physics who would self-identify as “a scientist” or, for that matter, even as “a physicist.” It is overwhelmingly more likely that he would say “quantum chromodynamics” or “lepton number violation in electroweak gauge theories” or “self-ionization of the vacuum” or some such fun thing...

I think this has a lot to do with the fact that the area of security is too new and too fuzzy. However, my point here is that a little common sense goes a long way even at this stage of our industry development. In light of this, next time you meet “a security expert,” ask him what is his area of expertise. If the answer is “security”, run!

Finally, career advice for those new to information security: don’t be a generalist. If you have to be a security generalist, be a “generalist specialist;” namely, know a bit about everything PLUS know a lot about something OR know a lot about “several somethings.” If you ONLY know “a bit about everything,” you’d probably die hungry...

Those are interesting insights. I agree with Anton's characterization of the field as being "too new." Theoretical physics is well over a hundred years old, while digital security is about forty years old.

Jeff Snyder's Security Recruiter Blog posted two good stories recently. The first is Hiring: Why Some Security Jobs Go Unfilled:

I started thinking about why some jobs are open for so long or go unfilled entirely...

A company recently sent a Security Analyst / Security Engineer job description to me for my review. They’ve had the job posted to major job boards for months but can’t seem to find the right person. As I studied the description, I quickly recognized that they were looking for at least two and possibly three different skill sets that typically don’t fit together in one person’s resume.

I pondered why they would create such a difficult expectation that essentially set them up to fail in their quest to find the right security job candidate... [C]ompanies across the nation is a significant squeezing of the belt. CISOs are pressured to deliver more results with less resources. Security professionals have to wear more hats than ever before and they have to be great at nearly everything they do in order to capture the most appealing jobs...

Recruiters don’t create candidates, we find those who already exist. If the person a company wants to hire doesn’t exist or doesn’t exist very often, I may be staring at a search that is set up to fail.

I agree with that statement too, but this idea of wearing so many "hats" is a recipe for failure. Most security people can't keep up with one aspect of the industry, let alone multiple aspects. I wrote about this issue several years ago in More Unrealistic Expectations from CIOs when I raged against the idea of a "multitalented specialist."

My third post again comes from Jeff Snyder, in Conversation: With a CIO regarding his Security Staffing:

The CISO was explaining his company’s need to cut back on staffing levels... [S]omeone came up with the idea that this CIO's company could live with one less information security professional.

As of now, they have one security professional who does security analysis and project management work but not a lot of what he does is considered deeply hands-on technical work.

The other security professional on this CIO's staff is a hands-on technical professional who has very deep technical skills but he is not strong with regulatory compliance, risk management work or work that requires strong interpersonal skills...

My recruiting partner and the CIO came to the conclusion that both security professionals might have to go in order to hire someone who had a broader skill set that included both the business / risk / interpersonal skills and the deeply technical components all wrapped up in one person’s security / technology risk management skill set...

Security professionals in both the present and the future need to bring broad skill sets to prospective employers in order to satisfy the growing demands found in hiring manager’s job descriptions.

Wow. That is a recipe for disaster. Lay off two people who already understand the business in order to replace them with one newbie who is expected to do both jobs? Isn't that the unrealistic expectations problem cited in Jeff's first post?

Read More

2009 CDX Data Sets Posted

Earlier this year I posted Thoughts on 2009 CDX. Greg Conti just sent me a notice that the West Point Information Technology and Operations Center just published, for free, their Intrusion Detection Labeled Data Sets. They include packet captures generated by NSA Red Team activity, packet captures from West Point defenders, and Snort, DNS, Web server, and host logs. This is great data. Stop using the 1999 DARPA data sets. Please.

Read More

SANS Incident Detection Summit in DC in December

Last month I blogged about the SANS Forensics and Incident Response 2009 Summit Round-Up. I am pleased to announce that I will be working with SANS to organize a two day SANS Incident Detection Summit in DC in December. I am working on a preliminary agenda that includes two major themes: network-centric detection and host-centric detection. The Summit will include keynotes, practitioner briefings, tool briefings, vendor briefings, and panels.

As we develop the content I will report it here. I am excited about this event and look forward to seeing you in December. My goal is to "bring detection back", since we all know that detection never really died!

If there are topics you'd like to see at the Summit, feel free to share them here. Thank you.

Update: 9-10 December are the days for the Summit.

Read More

Review of IPv6 Security Posted

Amazon.com just posted my five-star review of IPv6 Security by Scott Hogg and Eric Vyncke. From the review:

I've read and reviewed three other books on IPv6 in the last four years: IPv6 Essentials, 2nd Ed (IE2E) in September 2006, Running IPv6 (RI) in January 2006, and IPv6 Network Administration (INA) in August 2005. All three were five-star books, but they lacked the sort of attention to security that I hoped would be covered one day. IPv6 Security by Scott Hogg and Eric Vyncke is the book for which we have been waiting. Although some of the early "philosophical" security discussions (what's a threat, where are they) are lacking, the overwhelming amount of thorough and actionable content makes this book a winner.

Read More

Blast from the Past

So why a picture of me in uniform from 2000? The answer lies in this article published last month titled Air Force Network Operations begins migration to centralized e-mail, network services:

The Air Force Chief of Staff Gen. Norton Schwartz signed a directive memorandum here recently granting the Air Force Network Operations commander centralized order-issue authority over the operation, defense, maintenance and control of Air Force networks.

As part of an ongoing service-wide cyber operations transformation, the Air Force will establish a centralized user directory and e-mail service known as ADX that will service all Air Force network users.

The changes will be relatively transparent to most network users, but this migration to centralized services will significantly improve security and efficiency on the Air Force Global Information Grid, officials said.

"Major commands and subordinate commanders will no longer 'own' networks, but will be responsible for their portion of the larger AF-GIG," General Schwartz stated in a mass e-mail to Airmen.

I find this fascinating because I was part of a group assembled in late 2000, a few months before I left the Air Force (early 2001), tasked with the same mission. Among other tasks, we were told to centralize Air Force email by June 2001. I think many people in the group photo are laughing hysterically because we knew it was impossible to accomplish anything in the time allotted.

The centralization process has apparently already started:

Keesler Air Force Base, Miss., is the first base to undergo the transformation with 1,800 out of 5,800 users already transferred. Over the next 18 months, the complete migration will include approximately 750,000 users at more than 240 locations around the world.

So, about 11 years after being told to accomplish the same task, the effort will be done! I think there are lessons here for anyone with a similarly large, bureaucratic, turf-centric, distributed, decentralized, global organization.

Read More