Version 1.2      2
    NIST S&P Version 2: The Big Two
    1.   “New” Big Data Security and 
         Privacy Design Patterns
    2.   Big Data Security Fabric
                                  Version 1.2      3
                                    Version 1.2       4
    Reality Check in Apache Ecosystem
    • Secure, Multi-Tenant Deployment
    • Much like the early days of Hadoop, Apache Storm originally evolved in an environment where security 
     was not a high-priority concern. Rather, it was assumed that Storm would be deployed to environments 
     suitably cordoned off from security threats. While a large number of users were comfortable setting up 
     their own security measures for Storm, this proved a hindrance to broader adoption among larger 
     enterprises where security policies prohibited deployment without specific safeguards.
    • Yahoo! hosts one of the largest Storm deployments in the world, and the engineering team recognized 
     the need for security early on, so it implemented many of the features necessary to secure its own 
     Apache Storm deployment. Yahoo!, Hortonworks, Symantec, and the broader Apache Storm community 
     have been working on integrating those security innovations into the main Apache code base.
    • That work is nearing completion, and is slated to be included in an upcoming Apache Storm release. 
     Some of the highlights of that release include:
    • Kerberos Authentication with Automatic Credential Push and Renewal
    • Multi-Tenant Scheduling
    • Secure integration with other Hadoop Projects (such as ZooKeeper, HDFS, HBase, etc.)
    • User isolation (Storm topologies run as the user who submitted them)
    • In the future, you can expect to see further integration between Apache Storm and security-focused 
     projects like Apache Argus (formerly XA Secure). http://bit.ly/1Dlf2UP 
                                         Version 1.2          5
     Implications | Directions
     • NIST Big Data PWG documentation should show 
      awareness of trends & current efforts (good & bad)
     • NIST Big Data PWG should be a step or two ahead
     • Incorporate or link to work in grid, VLDB, distributed 
      computing
     • May need to separate “Expository” from “Technical” 
      documents (a la Oasis TCs) 
     • What elements s/b fabric?
     • What elements s/b design patterns?
                                                            Version 1.2                   6
      Security & Privacy (& Management)
                                      System Orchestrator
         r         Big Data Application Provider
         e                                                                                   r
         d                                                                                   e
         i                                                                                 
         v                                                                                a  m
         o                                                                                t  u
         r                                                Visualizatio                    a  s
         P   DAT                                           Visualizatio             DAT   D  n
          
         a    A     Collection    Curation    Analytics             n    Access      A       o
         t     S     Collection   Curation     Analytics             n   Access       S      C
         a
         D     W                                                                      W
                                                     TT  SW
                                                     AAAA
                                                     DD
                   Big Data Framework Provider
                    Processing Frameworks (analytic tools, etc.) 
                       Horizontally Scalable                                                      t
                                                            Vertically Scalable                   n
                    Platforms (databases, etc.)                                                &  e
                                                                                                  m
                        Horizontally Scalable                                                  y
                                                                                               t ye
                                                            Vertically Scalable                i cg
                                                                                               r aa
                    Infrastructures                                                            u vn
                                                                                               c i
                        Horizontally Scalable (VM clusters)                                    e ra
                                                            Vertically Scalable                S PM
                    Physical and Virtual Resources (networking, computing, etc.)
                                                                                                    6