Testing

BeeWare with Russell Keith-Magee - Episode 64

Summary

When you have good tools it makes the work you do even more enjoyable. Russel Keith-Magee has been building up a set of tools that are aiming to let you write graphical interfaces in Python and run them across all of your target platforms. Most recently he has been working on a capstone project called Toga that targets the Android and iOS platforms with the same set of code. In this episode we explored his journey through programming and how he has built and designed the Beeware suite. Give it a listen and then try out some or all of his excellent projects!

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • We are also sponsored by Sentry this week. Stop hoping your users will report bugs. Sentry’s real-time tracking gives you insight into production deployments and information to reproduce and fix crashes. Check them out at getsentry.com and use the code podcastinit to get a $50 credit!
  • Visit our site to subscribe to our show, sign up for our newsletter, read the show notes, and get in touch.
  • To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we’re interviewing Russel Keith-Magee about the Beeware project, which is a collection of tools and libraries that are meant to be composed together for building up your Python development environment.

Interview with Firstname Lastname

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is the BeeWare project and what goals do you have for it? – Tobias
  • What kinds of projects are contained under the BeeWare umbrella and what inspired you to start creating these kinds of tools? – Tobias
  • Did each project arise from a particular need that you had at the time or has there been a logical progression from one tool to the next? – Tobias
  • At PyCon US of this year (2016) you made a presentation about the work that you have been doing to bring Python to the iOS and Android platforms. Can you provide a high-level overview for anyone who hasn’t seen that talk yet? – Tobias
  • Let’s talk about Toga – how does Toga differ from some of the other cross platform UI framework efforts for various languages like Kivy or Shoes? – Chris
  • What are some of the biggest challenges that you had to overcome in order to get Python to run on both iOS and Android? – Tobias
  • How does runtime performance for applications written in Python compare with the same program running in the languages that are natively supported on those platforms? – Tobias
  • Can you walk us through the low level flow of a single toga API request? – Chris
  • Do you view your work on Toga and the associated libraries as a hobby project or do you think that it will turn into a production ready tool set that people will use for shipping applications? – Tobias
  • IDEs like Android Studio and XCode have a lot of features that simplify the development and UI creation process. Do you have to forego those niceties when developing a mobile app in Python? – Tobias
  • Shipping Python applications is a problem that tends to pose a host of issues for people, which you are addressing with the Briefcase project. What are some of the biggest hurdles and design choices that you have encountered while working on that? – Tobias
  • Do you think that there will ever be a release of iOS or Android, or even a brand new mobile platform, that will ship with native Python support? – Tobias

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Bandit with Tim Kelsey, Travis McPeak, and Eric Brown - Episode 62

Summary

Making sure that your code is secure is a difficult task. In this episode we spoke to Eric Brown, Travis McPeak, and Tim Kelsey about their work on the Bandit library, which is a static analysis engine to help you find potential vulnerabilities before your application reaches production. We discussed how it works, how to make it fit your use case, and why it was created. Give the show a listen and then go start scanning your projects!

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project. And they just doubled the RAM for their introductory level servers, so that $20 will get you even more performance.
  • We are also sponsored by Sentry this week. Stop hoping your users will report bugs. Sentry’s real-time tracking gives you insight into production deployments and information to reproduce and fix crashes. Check them out at getsentry.com and use the code podcastinit at signup to get a $50 credit!
  • Visit our site to subscribe to our show, sign up for our newsletter, read the show notes, and get in touch.
  • To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we’re interviewing Tim Kelsey and Eric Brown about Bandit which is a static analysis engine for finding security vulnerabilities in your Python code.

Interview with Eric Brown, Travis McPeak and Tim Kelsey

  • Introductions
  • How did you get introduced to Python? – Chris
  • What is Bandit and what was the inspiration for creating it? – Tobias
  • How did you each get involved with the Bandit project? – Tobias
  • At what stage of the development process would you want to use Bandit? – Tobias
  • What kinds of analysis does Bandit do on the source code that it is run against? – Tobias
  • How does it determine whether a particular segment of code is introducing a vulnerability and what means does it use to determine the severity? – Tobias
  • What does the generated report include and what can be done with that information? – Tobias
  • What are some of the biggest design and implementation difficulties that have been encountered in the process of creating Bandit? – Tobias
  • How does bandit compare to similar tools in other languages such as Ruby’s BrakeMan? – Tobias
  • What are some of the most interesting extensions that you have seen for Bandit? – Tobias
  • What is on the roadmap for the future of Bandit? – Tobias

Keep In Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Buildbot with Pierre Tardy - Episode 57

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

As technology professionals, we need to make sure that the software we write is reliably bug free and the best way to do that is with a continuous integration and continuous deployment pipeline. This week we spoke with Pierre Tardy about Buildbot, which is a Python framework for building and maintaining CI/CD workflows to keep our software projects on track.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show, subscribe, join our newsletter, check out the show notes, and get in touch you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • We are also sponsored by Rollbar this week. Rollbar is a service for tracking and aggregating your application errors so that you can find and fix the bugs in your application before your users notice they exist. Use the link rollbar.com/podcastinit to get 90 days and 300,000 errors for free on their bootstrap plan.
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Pierre Tardy about the Buildbot continuous integration system.

Interview with Pierre Tardy

  • Introductions
  • How did you get introduced to Python? – Chris
  • For anyone who isn’t familiar with it can you explain what Buildbot is? – Tobias
  • What was the original inspiration for creating the project? – Tobias
  • How did you get involved in the project? – Tobias
  • Can you describe the internal architecture of Buildbot and outline how a typical workflow would look? – Tobias
  • There are a number of packages out on PyPI for doing subprocess invocation and control, in addition to the functions in the standard library. Which does buildbot use and why? – Chris
  • What makes Buildbot stand out from other CI/CD options that are available today? – Tobias
  • Scaling a large CI/CD system can become a challenge. What are some of the limiting factors in the Buildbot architecture and in what ways have you seen people work to overcome them? – Tobias
  • Are there any design or architecture choices that you would change in the project if you were to start it over? – Tobias
  • If you were starting from scratch on implementing buildbot today, would you still use Python? Why? – Chris
  • What are some of the most difficult challenges that have been faced in the creation and evolution of the project? – Tobias
  • What are some of the most notable uses of Buildbot and how do they uniquely leverage the capabilities of the framework? – Tobias
  • What are some of the biggest challenges that people face when beginning to implement Buildbot in their architecture? – Tobias
  • Does buildbot support the use of docker or public clouds as a part of the build process? – Chris
  • I know that the execution engine for Buildbot is written in Twisted. What benefits does that provide and how has that influenced any efforts for providing Python 3 support? – Tobias
  • Does buildbot support build parallelization at all? For instance splitting one very long test run up into 3 instances each running a section of tests to cut build time? – Chris
  • What are some of the most requested features for the project and are there any that would be unreasonably difficult to implement due to the current design of the project? – Tobias
  • Does buildbot offer a plugin system like Jenkins does, or is there some other approach it uses for custom extensions to the base buildbot functionality? – Chris
  • Managing a reliable build pipeline can be operationally challenging. What are some of the thorniest problems for Buildbot in this regard and what are some of the mechanisms that are built in to simplify the operational characteristics? – Tobias
  • What were some of the challenges around supporting slaves running on platforms with very different environmental characteristics like Microsoft Windows? – Chris
  • What is on the roadmap for Buildbot? – Tobias

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Hypothesis with David MacIver - Episode 52

Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.

Summary

Writing tests is important for the stability of our projects and our confidence when making changes. One issue that we must all contend with when crafting these tests is whether or not we are properly exercising all of the edge cases. Property based testing is a method that attempts to find all of those edge cases by generating randomized inputs to your functions until a failing combination is found. This approach has been popularized by libraries such as Quickcheck in Haskell, but now Python has an offering in this space in the form of Hypothesis. This week, the creator and maintainer of Hypothesis, David MacIver, joins us to tell us about his work on it and how it works to improve our confidence in the stability of our code.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
  • Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
  • Open Data Science Conference on May 21-22nd in Boston. 20%
  • Your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing David MacIver about the Hypothesis project which is an advanced Quickcheck implementation for Python.

Interview with David MacIver

  • Introductions
  • How did you get introduced to Python? – Chris
  • Can you provide some background on what Quickcheck is and what inspired you to write an implementation in Python? – Tobias
  • Are there any ways in which Hypothesis improves on the original design of Quickcheck? – Tobias
  • Can you walk us through the execution of a simple Hypothesis test to give our listeners a better sense for what Hypothesis does? – Chris
  • Have you had trouble getting people to use Hypothesis? How has adoption been? – David
  • What does this sort of testing get you that conventional testing doesn’t? – David
  • Why do you think this sort of testing hasn’t caught on in the Python world before? – David
  • Are there any facilities of the Python language that make your job easier? Are there aspects of the language that make this style of testing more difficult? – Tobias
  • What are some of the design challenges that you have been presented with while working on Hypothesis and how did you overcome them? – Tobias
  • Given that testing is an important part of the development process for ensuring the reliability and correctness of the system under test, how do you make sure that Hypothesis doesn’t introduce uncertainty into this step? – Tobias
  • Given the sophisticated nature of the internals of Hypothesis, do you find it difficult to attract contributors to the project? – Tobias
  • A few months ago you went through some public burnout with regards to open source and Hypothesis in particular, but circumstances have brought you back to it with a more focused plan for making it sustainable. Can you provide some background and detail about your experiences and reasoning? – Tobias
  • What’s next for Hypothesis? – Chris

Keep In Touch

Picks

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Al Sweigart on Python for Non-Programmers - Episode 19

Visit our site to listen to past episodes, learn more about us, and support the show.

Summary

We got the opportunity to speak with Al Sweigart about his work on books like ‘Automate The Boring Stuff With Python’ and ‘Invent With Python’. We discussed how Python can be useful to people who don’t work as software engineers, why coding literacy is important for the general populace and how that will affect the ways in which we interact with software.

Brief Introduction

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • Subscribe on iTunes, Stitcher, TuneIn or RSS
  • Follow us on Twitter or Google+
  • Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
  • I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at
  • We are recording today on July 27th, 2015 and your hosts as usual are Tobias Macey and Chris Patti
  • Today we are interviewing Al Sweigart about Python for non-programmers

Interview with Al Sweigert

  • Introductions
  • How did you get introduced to Python?
    • Started in PHP/Perl, introduced to Python in 2006
    • Lack of curly braces took some getting used to
    • Clarity of standard library was refreshing
  • What inspired you to start writing books for non-programmers?
    • Friend who took care of 10 year old interested in programming
    • Lack of coherent introductory material
    • Started writing a tutorial which grew to book length
    • All books published under Creative Commons license
  • You have written a few books about teaching Python to people who have never programmed, can you share your thoughts on the best order in which to introduce the various aspects of programming?
  • Where does software testing come in when teaching new coders how to program?
    • Use the logger, debugger, and assertions effectively
  • In invent with Python you use games as the vehicle to discuss the principles involved with writing code. What is it about computer games that makes them so popular as a means to introduce programming to newcomers?
    • Something everyone is familiar with
    • Easy to make a simple game to get started
    • Good way to get creative with programming
  • For automate the boring stuff with Python you focused on explaining how programming can be useful even if it is not someone’s occupation. How did you determine which kinds of activities to focus on for the book?
    • Got the idea at a meetup talking to someone who works in an office doing repetitive tasks
    • A lot of office jobs that involve tedious computer work which could be automated
  • What are your thoughts on the need for software literacy among the general population?
    • How much programming knowledge do you think is sufficient for a member of our modern society?
  • You also wrote about using Python to decrypt simple ciphers as a means to learn about code. What was the inspiration for this approach to software education?
    • One of the projects in invent with Python was a simple cypher, inspired further interest in the subject
  • In episode 7 with Jacob Kaplan-Moss we talked about how we define what a programmer is. Can you share your opinions on what separates someone who can understand code from someone who is a programmer?
    • Barriers to entry have been significantly lowered, making the distinction very fuzzy
    • Definition of programmer is becoming much wider
  • Books available at:

Picks

Keep In Touch

Holger Krekel on Py.Test - Episode 16

Visit our site to listen to past episodes, learn more about the show and sign up for our mailing list.

Summary

In this episode we talked to Holger Krekel about the py.test library. We discussed the various styles of testing that it supports, the plugin system and how it compares to the unittest library. We also reviewed some of the challenges around packaging and releasing Python software and our thoughts on some ways that they can be improved.

Brief Introduction

  • Welcome to Podcast.__init__ the podcast about Python and the people who make it great
  • Date of recording – July 8th, 2015
  • Hosts Tobias Macey and Chris Patti
  • Follow us on iTunes, Stitcher or TuneIn
  • Give us feedback on iTunes, Twitter, email or Disqus)
  • We donate our time to you because we love Python and its community. If you would like to return the favor you can send us a donation}. Everything that we don’t spend on producing the show will be donated to the PSF to keep the community alive.
  • Overview – Interview with Holger Krekel about his work on Pytest

Interview with Holger Krekel

  • Introductions
    • Programming for 25 years
    • Runs a consultancy
    • Been to almost every EuroPyCon and PyCon US
  • How did you get introduced to Python? – Chris
    • Wanted to write an HTTP proxy and Java I/O was too confusing. Jython took less than a day to get it working after 2-3 days on it with Java.
  • What inspired you to create Pytest, and how did the existing unittest framework play into the story? – Chris
    • Introduced to agile methods through the Zope community
    • Zope used unittest – didn’t like the boiler plate
    • Not in the spirit of Python
    • Only took ~200 lines of code to get a testing tool working
    • Original name was ‘utest’ – 2003
    • Pytest name came in 2004 on Pypy project
    • Huge number of tests on that project (20,000) – distributed test runner – xdist helped solve this.
  • There are many different styles of testing, such as BDD, unit testing, integration testing, functional testing, what attributes of py.test make it suitable or unsuitable for these different approaches? – Tobias
    • What are your views on black box testing and how would someone use py.test to implement this approach? – Tobias
    • Pytest’s plugin architecture enables you to hook into the various phases of test execution enabling you to extend Pytest in all kinds of ways beyond the original design.
    • I have been hearing a lot about property based testing which was popularized by the Quickcheck module in Haskell. Does py.test support anything like that? – Tobias
    • hypothesis-pytest
  • Do you think the characteristics and nature of the unit testing framework being used have any effect on the number and quality of the tests developers write? – Chris
    • Developers find writing tests in Pytest to be fun compared to unittest
    • Which will help people write better tests
    • Encourages refactoring
  • Is there ever a time when you would advice _against_ writing tests? – Tobias
    • When exploring a problem, writing tests first doesn’t make sense
    • When getting feedback on a potential approach, writing tests first can be a waste of time
  • What are some signs that you watch out for when writing tests that tell you that a particular feature needs to be refactored? – Tobias
    • When the test code is fragile it should be refactored
    • Requires experience to really understand when to refactor
    • When it’s not fun anymore or the tests are repetitive
  • For someone who is converting their existing unit tests from UnitTest/Nose style to use py.test in an idiomatic manner, what are some of the biggest differences to be aware of? – Tobias
    • Generator/yield based testing should move to property based testing
    • If py.test can’t run a UnitTest/Nose style test it is considered a bug and gets fixed
  • Has the strict backwards compatibility policy presented any interesting technical challenges thus far? – Chris
    • Yes it definitely makes more work
    • However breaking the API in a large project like this will cause too many problems for users
  • py.test supports execution of tests written with other frameworks, how much ongoing maintenance does this feature require as changes are made to the other implementations? – Tobias
  • The web page says that Pytest is designed to work with domain specific and non Python tests, and in fact a coworker is using it to test a node.js project – how did Pytest’s design enable this? – Chris
    • Pytest uses a collection tree model to represent your project
      • This is not Python specific
      • All classes and functions are just mapped into this tree, not directly on the Python function
    • There are few Python specific hooks for fixtures etc.
    • People have written plugins so they can express their tests in YAML, Microsoft Excel
    • Tests are represented as items
    • All plugins are written in Python
  • What are some of the most interesting applications of py.test that you have seen? – Tobias
  • Speaking about adoption, do you have any sense of the relative adoption of Pytest versus unitest or other tools? – Tobias
    • Very hard to actually know
    • Download numbers are not a clear indicator due to robots, CI systems, etc.
    • Quantifying market share is hard to do
    • Popularity is not a useful heuristic in determining a good fot for technology adoption
      • But popularity is an indicator for the level of support you might receive
      • Tech can be popular but very poorly maintained
  • Are there any features of py.test that would make it suitable for use with configuration management tools and infrastructure testing? – Tobias
    • Example driven testing
    • Run py.test from a blackbox approach
    • Largest benefit would be from having one testing tool used across the organization
  • Where do you see Pytest and more generally test frameworks headed in the future? – Chris
  • Any questions we didn’t ask?
    • Pytest is a very healthy project! There are 10 regular contributors – this is exceptional among OSS projects

Picks

Keep In Touch

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA