Human Behavior Analysis in Uncontrolled Environmentscan be categorized in two main challenges:1) Feature extraction and 2) Behavior analysisfrom a set of corporal language vocabulary. Inthis work, we present our achievements characterizingsome simple behaviors from visual data ondifferent real applications and discuss our plan forfuture work: low level vocabulary definition frombag-of-gesture units and high level modelling andinference of human behaviors.