Releasing Your First (Python) Open Source Project to the Masses!

This Hackin Cast will build somewhat on the "Intro to Git" workshop I presented at WWHF 2020, and continue the journey of taking your idea for a cool new project, and moving that forward to actually getting it out in front of the world. You will benefit from either having seen that workshop, or having done any of the many Git tutorials available online (e.g. https://try.github.io/)., We'll pick up from having just learned how to start using Git, and work through how to take that knowledge and starting your own first open source project. We'll start a new project in Python, talk through how to build that project, package it up, build some testing around it, and get it pushed up to GitHub where others in the world can start finding it and making use of it. This Hackin cast is appropriate for attendees of all levels, and no prior knowledge (other than very basic command line and git usage) will be expected.

Transcript

1. LLNL-PRES-698283
   This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory
   under contract DE-AC52-07NA27344. Lawrence Livermore National Security, LLC
   Releasing Your First (Python)
   Open Source Project to the Masses!
   Wild West Hackin’ Cast
   2021-01-13 Ian Lee
   @IanLee1521
   

   View Slide

2. LLNL-PRES-698283
   2
   ▪ Part 1: Level setting
   — Quick primer on Git (see WWHF workshop for deeper dive)
   ▪ Part 2: Applying that to existing project
   — Small improvements matter!
   ▪ Part 3: Gitting out on your own
   — Starting your own project
   Schedule
   

   View Slide

3. LLNL-PRES-698283
   3
   Can’t I just have the version in the name?
   https://www.datamation.com/news/tech-comics-version-control-1.html
   

   View Slide

4. LLNL-PRES-698283
   4
   Common Version Control Tools
   

   View Slide

5. LLNL-PRES-698283
   5
   Fear of Git
   If that doesn't fix it, git.txt contains the
   phone number of a friend of mine who
   understands git. Just wait through a few
   minutes of 'It's really pretty simple, just
   think of branches as...' and eventually
   you'll learn the commands that will fix
   everything.
   https://xkcd.com/1597/
   

   View Slide

6. LLNL-PRES-698283
   6
   Staged vs not staged vs untracked
   https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository
   

   View Slide

7. LLNL-PRES-698283
   7
   Good Commit Messages
   Merge branch 'asdfasjkfdlas/alkdjf' into
   sdkjfls-final
   https://xkcd.com/1296/
   

   View Slide

8. LLNL-PRES-698283
   8
   A Word About Branch Names
   https://www.bcadoption.com/resources/articles/adoption-friendly-family-trees
   

   View Slide

9. LLNL-PRES-698283
   9
   DEMO
   Working on a Git repo with a remote
   https://git-school.github.io/visualizing-git/#free-remote
   Working with Remotes
   

   View Slide

10. LLNL-PRES-698283
    10
    Git Flow
    https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
    

    View Slide

11. LLNL-PRES-698283
    11
    Git Flow: Main Branches
    https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
    

    View Slide

12. LLNL-PRES-698283
    12
    Git Flow: Feature Branches
    https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
    

    View Slide

13. LLNL-PRES-698283
    13
    Git Flow: Release Branches
    https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
    

    View Slide

14. LLNL-PRES-698283
    14
    Git Flow: Maintenance Branches
    https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow
    

    View Slide

15. LLNL-PRES-698283
    15
    Working with remotes
    

    View Slide

16. LLNL-PRES-698283
    16
    DEMO
    Working on a Git repo with a remote with upstream changes
    https://git-school.github.io/visualizing-git/#upstream-
    changes
    Working with Upstream Changes
    

    View Slide

17. LLNL-PRES-698283
    17
    Let’s Git To It!
    

    View Slide

18. LLNL-PRES-698283
    18
    Code Hosting Platforms
    

    View Slide

19. LLNL-PRES-698283
    19
    ▪ Reading
    https://adhdproject.github.io/#!WWHF/2020/Deadwood/Intro_WWHF2020_Deadw
    ood.md and found a link that isn’t rendering
    ▪ https://github.com/adhdproject/adhdproject.github.io/blob/master/WWHF/2020/D
    eadwood/Intro_WWHF2020_Deadwood.md
    WWHF 2020 ADHD Labs
    

    View Slide

20. LLNL-PRES-698283
    20
    ▪ https://github.com/activecm/rita
    ▪ Reading the documentation, found a bug in the docs
    (https://github.com/activecm/rita/blob/master/docs/Docker%20Usage.md#running-
    rita-with-docker-compose)
    ACM RITA
    

    View Slide

21. LLNL-PRES-698283
    21
    Click Here
    https://docs.docker.com/get-started/overview/
    

    View Slide

22. LLNL-PRES-698283
    22
    

    View Slide

23. LLNL-PRES-698283
    23
    

    View Slide

24. LLNL-PRES-698283
    24
    Committing Your Changes In The UI
    

    View Slide

25. LLNL-PRES-698283
    25
    ▪ https://github.com/activecm/rita
    — Documentation updates?
    ▪ https://github.com/gentilkiwi/mimikatz
    — Add LICENSE file?
    ▪ https://github.com/rapid7/metasploit-framework
    — Add a new exploit?
    — Fix a bug in an existing exploit?
    ▪ https://github.com/byt3bl33d3r/CrackMapExec
    — Maybe consider adding some CI testing ?
    Security Tools
    

    View Slide

26. LLNL-PRES-698283
    26
    ▪ https://github.com/adhdproject/awesome-active-defense
    ▪ https://github.com/juliocesarfort/public-pentesting-reports
    ▪ https://github.com/sbilly/awesome-security
    ▪ https://github.com/onlurking/awesome-infosec
    ▪ https://github.com/joe-shenouda/awesome-cyber-skills
    ▪ https://github.com/fabacab/awesome-cybersecurity-blueteam
    ▪ https://github.com/meirwah/awesome-incident-response
    ”Awesome” Lists
    

    View Slide

27. LLNL-PRES-698283
    27
    Let’s Git Out There
    

    View Slide

28. LLNL-PRES-698283
    28
    ▪ There are many choices out there…
    ▪ Don’t let wizards decry your preferences, just find something that works for you!
    ▪ For me:
    — VS Code (https://code.visualstudio.com/)
    • Remote Development Toolkit (https://code.visualstudio.com/docs/remote/remote-overview)
    — Windows 10
    • WSL 2 (https://docs.microsoft.com/en-us/windows/wsl/install-win10)
    — macOS / Linux
    • Homebrew (https://brew.sh/)
    — Also…
    • ZSH + Oh My Zsh (https://ohmyz.sh/)
    • Docker (https://www.docker.com/)
    Kickstarting Your Development
    

    View Slide

29. LLNL-PRES-698283
    29
    ▪ Windows 10 + WSL 2 (Ubuntu 20.04) + Docker + VS Code
    ▪ Spin up a new docker container
    — docker run -it –v $PWD:/code python:slim /bin/bash
    ▪ Connect to it with VS Code + Remote Development
    ▪ Proof* that there is no Tom-foolery happening that you don’t see!
    Setup
    

    View Slide

30. LLNL-PRES-698283
    30
    Kickstarting a Project
    ▪ python3 -m pip install --user poetry
    — Add `$HOME/.local/bin` to your $PATH
    ▪ poetry new --name awesome
    ▪ Pull up the docs:
    — https://python-poetry.org/docs/
    awesome-project/
    ├── README.rst
    ├── awesome
    │ └── __init__.py
    ├── pyproject.toml
    └── tests
    ├── __init__.py
    └── test_awesome.py
    

    View Slide

31. LLNL-PRES-698283
    31
    Minor updates
    ▪ Make it a git repo (`git init`)
    ▪ Update “authors”
    — “Ian Lee "
    ▪ Add some dependencies
    — poetry add requests
    — poetry add –D black flake8
    — poetry update
    ▪ https://python-
    poetry.org/docs/pyproject/
    

    View Slide

32. LLNL-PRES-698283
    32
    What Now?
    ▪ Add a basic command line tool
    ▪ Let’s add a new Python module
    ▪ Run `poetry install` to update the
    environment
    

    View Slide

33. LLNL-PRES-698283
    33
    Build and Publish the Package
    $ poetry build
    Building awesome (0.1.0)
    - Building sdist
    - Built awesome-0.1.0.tar.gz
    - Building wheel
    - Built awesome-0.1.0-py3-none-any.whl
    ▪ $ poetry publish
    https://pypi.org
    

    View Slide

34. LLNL-PRES-698283
    34
    Other Additions
    ▪ Publish the Git repo
    ▪ Add unittests
    ▪ Add Continuous Integration
    ▪ Add documentation
    — Standalone? Website? README only?
    

    View Slide

35. LLNL-PRES-698283
    35
    ▪ See also Marcello's awesome Pretty Little Python Secrets BHIS Webcasts last year
    — Part 1: Installing Python Tools/ Libraries the Right Way
    • https://www.youtube.com/watch?v=ieyRV9zQd2U
    — Part 2: Python Development & Packaging as Beautiful as a Poem
    • https://www.youtube.com/watch?v=tNlurLxcf68
    See also
    

    View Slide

36. LLNL-PRES-698283
    36
    ▪ https://pages.github.com/
    ▪ Easy way to start a website
    ▪ Uses Jekyll (https://jekyllrb.com/)
    ▪ All on top of Git, Hosted by GitHub
    ▪ Example:
    — https://github.com/ianlee1521/ianlee1521.github.io
    — Becomes https://ianlee1521.com
    Build your own website with GitHub Pages
    

    View Slide

37. LLNL-PRES-698283
    37
    ▪ I strongly encourage you to start any code / documentation / note projects with
    `git init `
    ▪ Even if you NEVER intend to share it with anyone, anywhere, ever!
    ▪ You don’t have to be a “1337 haxor dev” to make a meaningful impact on a project.
    Anything!
    

    View Slide

38. Leave things better than you found them.
    Thank You!
    $ cat git.txt
    Ian Lee – @IanLee1521
    Disclaimer
    This document was prepared as an account of work sponsored by an agency of the United States government. Neither
    the United States government nor Lawrence Livermore National Security, LLC, nor any of their employees makes any
    warranty, expressed or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or
    usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe
    privately owned rights. Reference herein to any specific commercial product, process, or service by trade name,
    trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or
    favoring by the United States government or Lawrence Livermore National Security, LLC. The views and opinions of
    authors expressed herein do not necessarily state or reflect those of the United States government or Lawrence
    Livermore National Security, LLC, and shall not be used for advertising or product endorsement purposes.
    

    View Slide

39. LLNL-PRES-698283
    39
    ▪ Computer Engineer
    — 2010 – 2015: Primarily Software Development
    • Python, Web, (some) System Administration
    — 2016 – Present: Cyber Security
    • 2016 – Present: Cyber Assessment Coordinator
    – Red team on super computers
    • 2018 – Present: Alternate Organization ISSO
    ▪ @IanLee1521
    — twitter.com/IanLee1521
    — github.com/IanLee1521
    — speakerdeck.com/IanLee1521
    — Discord: IanLee1521
    ▪ “Leave things better than you find them”
    Who Am I ?
    

    View Slide

40. LLNL-PRES-698283
    40
    ▪ Git Tutorials
    — https://try.github.io/
    — http://learngitbranching.js.org/
    — https://www.atlassian.com/git/
    ▪ Commandline help
    — $ man git
    — $ git –help
    ▪ WWHF October 2020 Workshop
    — https://speakerdeck.com/ianlee1521/intro-to-git-for-security-professionals
    Getting Help
    

    View Slide

41. LLNL-PRES-698283
    41
    What is version control?
    http://smutch.github.io/VersionControlTutorial/
    

    View Slide