Social Media and Image Analysis

The streams of data that are available these days are staggering. There are more and more ways to mine social media in order to find original and thought provoking metrics. In some ways, I see remote sensing as its own social media stream, like each pixel is tweeting something about the world. Is it green, blue, has it changed, does it influence its neighboring pixels? Groups of pixels can comprise an object and those objects also can exist, “Hey, I’m green grass” or change, “I was dry grass last week, but I’ve greened up.” With collections over time, this information can predictably evolve e.g. “I was dry grass, then I became vegetation, then I died, then I greened up again.” If we attempt to model this pixel, it would predictably die again. Each pixel, object, image has something to say—and the world of social media can have something to say about imagery. Geotagged pictures, tweets, weather information, political events, and policy changes can all be tagged as additional content to information contained in imagery. And with that information, the ability to model and predict about the world around us becomes infinite.

 

Imagery can be thought in the same unstructured terms as social media—it’s another piece of the puzzle that may or may not take a predefined shape. Think about being able to mine imagery with hash tags—the presence of certain objects cues a tag or a change in an area cues another kind of tag. There is an abundance of new high-resolution sources of imagery from SkyBox, Urthecast and Planet Labs—especially video components. But is all that data really interesting? Can data from these and other instruments be searched for only what is interesting and catalogued by that information? An extreme example of this could be for security analysis. Take the Empire State Building--images collected every day, a couple videos, and show people coming and going. The building itself is unchanging. The change a security analyst would be interested in would be the sudden appearance of any kind box larger than a square foot. Once that change occurs, suddenly the previous images have value for showing the change as opposed to before when they just showed consistency. When that change occurs, the change is tagged and communicated. The concept is similar to a person who tweets constantly that she, “is sitting on the couch.” It isn’t really interesting until “sitting on the couch” becomes “My house is on fire and it’s spreading to others #Boulder”.

 

I read an interesting article on offering masters in DataScience on GNIP’s blog https://blog.gnip.com/data-science-masters/. GNIP is an organization specializing in delivering APIs for social media feeds. In many ways remote sensing has tackled the difficulty of being a scientist studying phenomena like botany who needs math skills, computer science skills, and visualization skills in order to use all the tools available to a botanist. The remote sensing community has much to offer the world of big data by the nature of the disciple, but it must also be on the receptor end—integrating data from outside of itself. Community remote sensing, CRS, is nothing new. This Article from Annelie Schoenmaker defines CRS as, “Location technology that combines remote sensing with citizen science, social networks and crowd-sourcing to enhance the data obtained from traditional sources. It includes the collection, calibration, analysis, communication or application of remotely sensed information by these community means.” At present, the remote sensing big data people and other big data people might mingle at a conference, but the connections are still being forged.

 

With data feeds from Twitter and other social media sources being mineable, the contextual information that can be discovered about imagery is infinite; it’s a matter of asking the right questions. What questions should remote sensing, earth science, and defense and intelligence be asking social media to enhance information that is already contained in imagery? Can this information be used to either predict what will happen next orto forensically understand what has just happened? Who is thinking about this in the remote sensing community? How can these connections be fostered? How does image analysis software need to integrate with social media feeds? If I come up with more thoughts, I’ll tweet about it. My request is that you do too. @asoconnor