Easy app metrics with Pup

This was originally posted on datadoghq.com by Vivek Patel (2012 hackNY Fellow).

This is a guest-post from Vivek Patel, who joined Datadog as a HackNY fellow this summer. College students wondering about their options next summer may want to read this article as well.

Why pup?

One of my main fears as a developer is users complaining about slow page loads. Loading times directly affect both user experience and my bottom line, so I need a way to dig into the performance of my application to address this issue.

What are my options?

I can tail logs, but logs suck. The pile of numbers and text provide limited context and insight. Logs are more useful to identify errors. But in raw form, they’re impractical to observe the performance of my application. I need to spend time upfront to set log post-processing up and adding and extracting new metrics to and from logs quickly becomes a pain.

If only I had easy-to-use, free performance monitoring that gives me actionable graphs in 2 minutes…

That’s where Pup comes in.

Getting started

 

Pup

Pup is fully open-source. To install it, type this at the command-line:

$ sh -c "$(curl -L http://dtdg.co/setup-pup)" 

Then navigate to http://localhost:17125 and bam! Within seconds, you will see metrics streaming in real time.

Pup is designed for application metrics:

In addition to collecting system metrics, Pup faithfully collects and displays custom metrics. To do so it harnesses the power of StatsD, a metrics collector developed by Etsy. Pup comes with StatsD built-in; no need to reinvent the wheel.

To send custom metrics to Pup, you can use the dogstatsd library, which as you will see requires a grand total of 3 lines of code to instrument application code. This library comes in a number of flavors (python, ruby, php, etc.)

Here’s an example Ruby web application using Sinatra. First, we need to install the dogstatsd library:

# For ruby library
gem install dogstatsd-ruby

# For python library, either:
# Install in system python ...
sudo easy_install dogstatsd-python

# .. or into a virtual env
easy_install  dogstatsd_python

 

Now to add Pup instrumentation:

require 'sinatra'
require 'statsd'

statsd = Statsd.new()

get '/' do
  statsd.time('pup.render_time') do
    erb: index
  end
end

 

And you’re done. As soon as the page is hit, you’ll see something like this in Pup:

pup render time

That’s it! If you use Datadog, the first period creates a namespace to organize metrics under.

Here’s a 1 minute screencast summarizing the whole process:

Looking for more?

More thorough documentation on using DogstatsD can be found on api.datadoghq.com.

To correlate metrics and events across your infrastructure, give Datadog a spin. It only takes a few minutes to get started.

Become a Fall 2012 hackNY Student Hackathon Sponsor!

Since April 2010, hackNY’s student hackathons have attracted hundreds of students from more than 30 universities for 24-hour events in which participants collaborate on creative coding challenges. At the beginning of a hackNY student hackathon, New York City startups selected by the student organizing committee demo their APIs. Students then form teams to brainstorm ideas for projects to build based on these APIs, working through the night to turn their ideas into reality in time to present before a panel of judges the following day, competing for prizes and glory.

The fall 2012 hackNY student hackathon (our sixth!) will be September 29-30 at NYU’s Courant Institute of Mathematical Sciences. It will be an exciting, well-attended event and we’re looking for sponsors.

For more information on sponsorship and to discuss becoming a sponsor, please contact Team hackNY at [email protected]

hackNY Summer Series: Dan Huttenlocher, Greg Pass, and Thatcher Bell

In this post 2012 hackNY Fellow Jason Wright describes the hackNY Summer Series lecture by Dan Huttenlocher, Greg Pass, and Thatcher Bell.

Being the only Cornell student in this year’s class of hackNY Fellows, I was pretty excited when I noticed that Dan Huttenlocher, dean of computing and information science at Cornell, was giving a talk to hackNY along with Greg Pass, former CTO of Twitter and current Entrepreneurial Officer of CornellNYC Tech, and Thatcher Bell, Principal at DFJ Gotham, both Cornell alumni. To be honest, this was mostly because of (1) my Big Red pride, and (2) I was looking to convince them to let me hang out at Cornell’s new space at Google NYC as much as possible this fall.

However, what myself and the other fellows learned was that CornellNYC Tech is so much more than a satellite campus of our stodgy institution in Ithaca, NY, to be filled with traditional lecture halls and professors. Rather, it’s a visionary, one-of-a-kind proposal that seeks to integrate academia deeply with entrepreneurship, industry, and the technology community.

To give some background, CornellNYC Tech was selected in December 2011 by Mayor Bloomberg as the winner of a competition for a new applied sciences graduate campus in New York City, for which it received land on Roosevelt Island for construction to be completed in 2017. In the interim, Cornell will run classes & events out of temporary space in Google’s NYC office. Once the campus is open, Cornell will offer a joint two-year M.S. in Applied Science with the Technion Institute, in addition to traditional M.S. and Ph.D. offerings in fields like computer science, electrical engineering, and information science.

Huttenlocher began by describing the current model of tech development: a pipeline consisting of abstract academic research, focused R&D of concepts that come from universities, product development, marketing, and sales. The cordoning off of academic research means that ideas are disconnected from innovations, and tons of academic research doesn’t translate into meaningful technology because entrepreneurs either aren’t aware or can’t find profitable endeavors that use it.

CornellNYC Tech wants all of those stages to occur in parallel. The campus will break down traditional university conventions and create a place for students, faculty, industry leaders, and entrepreneurs to do research & make things together, at the same time. In doing so, the hope is that connections will naturally form between those groups. This will not only create mutual opportunities for collaboration, but will also enable real-time feedback loops to drive research & innovation in productive directions. Academics will advise entrepreneurs on technology development, and entrepreneurs will share real-world experiences and knowledge with academics.

In many ways, the project is itself a frighteningly ambitious startup — a proposal to disrupt academia by utilizing the help of government, two institutions not exactly known for their agility or flexibility. In addition, the inclusion of Technion as a partner institution flips the trend of U.S. universities expanding overseas on its head. The hope is that Technion, which is positioned at the center of the technology industry in Israel, will bring an international perspective to NYC.

Another important goal of the project is to accelerate the growth of the burgeoning tech industry in NYC. To accomplish this, students will have advisors in industry in addition to academic advisors. All three speakers specifically mentioned graduate education as something that should be closer to apprenticeship, in which students hone their skills and prepare to use them to do amazing things outside of school. Thatcher argued that this type of network-building was vital in creating a thriving a tech community to overcome the vastness of NYC’s economic activity in which other industries compete for people, attention, and investment.

Huttenlocher also addressed the issue of finding the right faculty members to participate. Much like a startup, he said, the first few hires set the tone for the organization’s entire culture. For Cornell, finding faculty members willing to relocate and work in an environment where they’ll be coworking right next to students and entrepreneurs instead of in a cozy office will be a challenge. While faculty members will still be affiliated with departments established in the Ithaca campus, their work in NYC will be much more decentralized and interdisciplinary. However, those people do exist — Deborah Estrin and Huttenlocher himself, who has bounced between Cornell, Xerox PARC, and various startups, are proof of that.

Greg Pass brings a wealth of experience to the table as well, having scaled Twitter from its early fail-whale days to the powerhouse that it is now. He emphasized that innovation should be data-driven and incremental, as opposed to academic theorizing that often privileges finding “clever” solutions at the expense of quick, practical ones. In response, Huttenlocher quipped that too much emphasis on being clever “bites you in the ass 100% of the time.”

All were sure to mention the experimental nature of the project, noting that we won’t see results for several years. If successful, it’s a model of education that could be applied in many other areas, even in primary education, and one that could revolutionize universities as we know them.

The excitement of hackNY in the room (the beautiful art.sy headquarters) was palpable, and I’m incredibly enthusiastic to have a future at the growing nexus of hackNY, CornellNYC Tech, and the NYC tech community.

hackNY Fellows Workshop: 3D Games in Unity

In this post, class of 2012 hackNY Fellow David Coss describes his workshop on 3d games in unity.

A week after Emmett’s workshop on game development using cocos2d and box2d, I gave my own workshop about making 3d games in unity. Unity is a free 3d game engine that allows one to build games for several different platforms, including desktop (windows/osx) and mobile (ios/android). The scripting framework is built on mono, an open source implementation of the .NET framework. Although the main language is c#, unity also supports unityscript (its own variant of javascript) and boo (a language inspired by python and developed for .NET/mono).

For the workshop, I decided to show hackNY Fellows how to build a simple first-person 3d platformer. Much of the workshop didn’t involve any coding at all, and instead focused on learning unity’s interface and object hierarchy, designing a level, and building basic game objects. After covering these concepts, we used scripting to animate the objects in our level and enhance their interactivity with the player character. Concepts covered in our scripting included basic transposition and rotation, referencing other game objects, and raycasting (which was used to detect whether the player character had entered a spotlight).

You can try out a (super janky) demo here, and download the source project here. You will need the unity web player to play the demo, but if you’ve downloaded unity you should already have it.

Students, here’s why you should do hackNY

This was originally posted on jessepollak.me by Jesse Pollak (2012 hackNY Fellow).

If you want to see the NYC tech scene flourish, share this piece with your networks and encourage any students you know to check out hackNY.

If you’re a student, read this and consider applying for the fellowship when applications open in September. I promise you that it is one of the best things you can do (according to one fellow, “I can CONFIRM that hackNY »»» Google”).

I’m Jesse Pollak, a student at Pomona College. This summer, I was lucky enough to participate in the hackNY fellows program. If you don’t already know, hackNY is a non-profit organization whose mission is to ”to federate the next generation of hackers for the New York innovation community.” In addition to holding twice annual student hackathons, each summer hackNY organizes the hackNY fellows program:

a program that pairs quantitative and computational students with startups which can demonstrate a strong mentoring environment: a problem for a student to work on, a person to mentor them, and a place for them to work. Students enjoy free housing together and a pedagogical lecture series to introduce them to the ins and outs of joining and founding a startup.

On Friday, the program culminated with the hackNY DemoFest (watch here) and on Saturday, I took the bus home for the summer. Accordingly, I thought I’d write a little about my experience, so the internet can know how amazing the program really is.

Living with 30 of the smartest student hackers is unreal

I go to school at Pomona College. There is no doubt that it is a great school, but being a liberal arts college, there isn’t much of a technology scene. Yes, our computer science department is amazing, and it’s merged with the Harvey Mudd department, but the kids I live and interact with on a day to day basis are for the most part not programmers and aren’t that interested in the startup world.

Moving into an apartment with 3 other ‘hackers,’ and having 28 more live down the hall, was a mind-blowing experience. Having 30 programmers within a few steps meant that there was always someone who could help me with a problem or collaborate on a project. And, since these were some smart kids, I felt like I was constantly learning something new. But, don’t trust me on their intelligence and abilities, check out some of the things they built this summer (mostly, in the time they weren’t working a full time job):

Pup
Heads Up Hot Dog
Dablooms
The Facebook Book
Anodyne
Facebook Nanny
Mentor.im
Treasure Hunter
Your Guess Is Wrong
Mr. Schemato

For even more, check out the DemoDay video.

Not only were they all extremely smart and helpful, they were also just a lot of fun. Yes, we spent a lot of time in front of our computers coding, but we always took the time to explore the city. We traveled in a pack on weekend nights to all of the “cool places,” usually being lead by one of the mentors who had been a fellow in the previous years. In fact, more often than not, I was overwhelmed by the dizzying array of social options presented by the fellows for a given weekend night.

New York City is unlike any other city

I grew up in Washington, DC, so I thought I knew what it was like to live in a city. Well, let me tell you that New York City is a whole different ball game. There is always something cool to do; whether you want to go to a tech meetup, a concert, a museum, or an art show, every night there is an absurd number of choices. And, the food. The food is unbelievably delicious and it’s always open. It’s hard to appreciate these benefits of NYC until you live there for awhile, then move somewhere else. Now that I’m home, the only thing I want to do is go back.

I went into the summer almost certain I would spend my next 3 summers in SF and move there after graduation. After living in NYC for 10 weeks, I’m certain I’ll be back.

The startups we work at are amazing

I got to work at BuzzFeed. There, I was given full control over building consumer facing features and was treated like a real member of the team. The people I was working with were unbelievably smart, the office space was beautiful, the free lunches, dinners, and breakfasts were delicious, and everyday I wanted the day to be longer than it was.

And, BuzzFeed wasn’t the only cool place. In fact, all of the startups fellows worked at were leaders in the startup world: 10gen Bit.ly, Art.sy, Tumblr, Sailthru, and Boxee just to name a few. And, unsurprisingly, at each company the fellow had an amazing experience.

No matter where we worked, we all owned our features, made new friends, learned new things, worked in loving environments, and felt like a real part of the team.

The job and living environment wasn’t even the coolest part

Meeting some of the smartest people in the NYC tech scene is. Two or three times a week, the 30 of us sat down for a casual conversation (over a free dinner) with some of the greatest minds in startups. We talked anonymity with moot, SOPA/PIPA with Alexis Ohanian, founding teams with Chris Dixon and investing with Fred Wilson and the Union Square Ventures partners. In 10 weeks, in addition to the above, we were lucky enough to have personal conversations with:

Zach Sims and Ryan Bubinski, the founders of Codecademy
Hursh, Cemre, and Josh, the founders of Branch (coming soon, awesome)
Jonah Peretti, the founder of HuffPo and BuzzFeed
Billy Chasen, the technical co-founder of Turntable.fm
Joanne Wilson, a prolific NYC writer and angel investor
Dennis Crowley and Harry Heyman, the co-founders of Foursquare
Peter Bell, the VP of Engineering at GA
Joel Spolsky, the founder of Fog Creek and Stack Overflow
Josh Knowles, the managing director of Pivotal Labs
Anthony Volodkin, the founder of Hype Machine
Thatcher Bell, the managing director of DFJGotham
Whitney Hess, an awesome UX designer
Chad Dickerson, the CEO of Etsy

You can learn a lot from reading Hacker News, but getting to ask whatever question you want to some of the smartest people around is an incomparable learning experience. Plus, all these people are a lot of fun to hang out with.

hackNY is pretty unbeatable

Living with some of the smartest students in the country; working at some of the coolest startups in NYC; meeting some of the most influential people in the startup scene.

This was my first summer in tech, but after talking with other fellows, I can honestly say that this is an unbeatable experience. Yes, Facebook or Google are great, but I promise hackNY is an experience worth having.

The NYC tech scene is growing extremely fast and in order for it to really flourish, we need as many talented engineers as possible. Programs like hackNY and the Turing Fellowship are taking real steps to providing this necessary resource. So, I urge you to apply for thehackNY fellowship program when applications open in the fall. Hopefully I’ll be back in the city next year as a mentor and we’ll get to hang!

Also, attend the hackNY Fall hackathon on September 29-30th.

p.s. if you enjoyed reading this, you should check out the rest of my blog and follow me on twitter.