DevOps

Unpacking The Python Toolkit For Chaos Engineering - Episode 199

Summary

Chaos engineering is the practice of injecting failures into your production systems in a controlled manner to identify weaknesses in your applications. In order to build, run, and report on chaos experiments Sylvain Hellegouarch created the Chaos Toolkit. In this episode he explains his motivation for creating the toolkit, how to use it for improving the resiliency of your systems, and his plans for the future. He also discusses best practices for building, running, and learning from your own experiments.

Announcements

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute. And don’t forget to thank them for their continued support of this show!
  • And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Sylvain Hellegouarch about Chaos Toolkit, a framework for building and automating chaos engineering experiments

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Chaos Engineering is?
  • What is the Chaos Toolkit and what motivated you to create it?
    • How does it compare to the Gremlin platform?
  • What is the workflow for using Chos Toolkit to build and run an experiment?
    • What are the best practices for building a useful experiment?
    • Once you have an experiment created, how often should it be executed?
  • When running an experiment, what are some strategies for identifying points of failure, particularly if they are unexpected?
    • What kinds of reporting and statistics are captured during a test run?
  • Can you describe how Chaos Toolkit is implemented and how it has evolved since you began working on it?
  • What are some of the most challenging aspects of ensuring that the experiments run via the Chaos Toolkit are safe and have a reliable rollback available?
  • What have been some of the most interesting/useful/unexpected lessons that you have learned in the process of building and maintaining the Chaos Toolkit project and community?
  • What do you have planned for the future of the project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Gnocchi: A Scalable Time Series Database For Your Metrics with Julien Danjou - Episode 189

Summary

Do you know what your servers are doing? If you have a metrics system in place then the answer should be “yes”. One critical aspect of that platform is the timeseries database that allows you to store, aggregate, analyze, and query the various signals generated by your software and hardware. As the size and complexity of your systems scale, so does the volume of data that you need to manage which can put a strain on your metrics stack. Julien Danjou built Gnocchi during his time on the OpenStack project to provide a time oriented data store that would scale horizontally and still provide fast queries. In this episode he explains how the project got started, how it works, how it compares to the other options on the market, and how you can start using it today to get better visibility into your operations.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so check out Linode. With 200 Gbit/s private networking, scalable shared block storage, node balancers, and a 40 Gbit/s public network, all controlled by a brand new API you’ve got everything you need to scale up. Go to pythonpodcast.com/linode to get a $20 credit and launch a new server in under a minute.
  • And to keep track of how your team is progressing on building new features and squashing bugs, you need a project management system designed by software engineers, for software engineers. Clubhouse lets you craft a workflow that fits your style, including per-team tasks, cross-project epics, a large suite of pre-built integrations, and a simple API for crafting your own. Podcast.__init__ listeners get 2 months free on any plan by going to pythonpodcast.com/clubhouse today and signing up for a trial.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Julien Danjou about Gnocchi, an open source time series database built to handle large volumes of system metrics

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what Gnocchi is and how the project got started?
    • What was the motivation for moving Gnocchi out of the Openstack organization and into its own top level project?
  • The space of time series databases and metrics as a service platforms are both fairly crowded. What are the unique features of Gnocchi that would lead someone to deploy it in place of other options?
    • What are some of the tools and platforms that are popular today which hadn’t yet gained visibility when you first began working on Gnocchi?
  • How is Gnocchi architected?
    • How has the design changed since you first started working on it?
    • What was the motivation for implementing it in Python and would you make the same choice today?
  • One of the interesting features of Gnocchi is its support of resource history. Can you describe how that operates and the types of use cases that it enables?
    • Does that factor into the multi-tenant architecture?
  • What are some of the drawbacks of pre-aggregating metrics as they are being written into the storage layer (e.g. loss of fidelity)?
    • Is it possible to maintain the raw measures after they are processed into aggregates?
  • One of the challenging aspects of building a scalable metrics platform is support for high-cardinality data. What sort of labelling and tagging of metrics and measures is available in Gnocchi?
  • For someone who wants to implement Gnocchi for their system metrics, what is involved in deploying, maintaining, and upgrading it?
    • What are the available integration points for extending and customizing Gnocchi?
  • Once metrics have been stored, aggregated, and indexed, what are the options for querying and analyzing the collected data?
  • When is Gnocchi the wrong choice?
  • What do you have planned for the future of Gnocchi?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

How Python Is Used To Build A Startup At Wanderu with Chris Kirkos and Matt Warren - Episode 183

Summary

The breadth of use cases that Python supports, coupled with the level of productivity that it provides through its ease of use have contributed to the incredible popularity of the language. To explore the ways that it can contribute to the success of a young and growing startup two of the lead engineers at Wanderu discuss their experiences in this episode. Matt Warren, the technical operations lead, explains the ways that he is using Python to build and scale the infrastructure that Wanderu relies on, as well as the ways that he deploys and runs the various Python applications that power the business. Chris Kirkos, the lead software architect, describes how the original Django application has grown into a suite of microservices, where they have opted to use a different language and why, and how Python is still being used for critical business needs. This is a great conversation for understanding the business impact of the Python language and ecosystem.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Matt Warren and Chris Kirkos and about the ways that they are using Python at Wanderu

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by describing what Wanderu does?
    • How is the platform architected?
  • What are the broad categories of problems that you are addressing with Python?
  • What are the areas where you chose to use a different language or service?
  • What ratio of new projects and features are implemented using Python?
    • How much of that decision process is influenced by the fact that you already have so much pre-existing Python code?
    • For the projects where you don’t choose Python, what are the reasons for going elsewhere?
  • What are some of the limitations of Python that you have encountered while working at Wanderu?
  • What are some of the places that you were surprised to find Python in use at Wanderu?
  • What have you enjoyed most about working with Python?
    • What are some of the sharp edges that you would like to see smoothed over in future versions of the language?
  • What is the most challenging bug that you have dealt with at Wanderu that was attributable in some sense to the fact that the code was written in Python?
  • If you were to start over today on any of the pieces of the Wanderu platform, are there any that you would write in a different language?
  • Which libraries have been the most useful for your work at Wanderu?
    • Which ones have caused you the most pain?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Managing Application Secrets with Brian Kelly - Episode 181

Summary

Any application that communicates with other systems or services will at some point require a credential or sensitive piece of information to operate properly. The question then becomes how best to securely store, transmit, and use that information. The world of software secrets management is vast and complicated, so in this episode Brian Kelly, engineering manager at Conjur, aims to help you make sense of it. He explains the main factors for protecting sensitive information in your software development and deployment, ways that information might be leaked, and how to get the whole team on the same page.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Brian Kelly about how to store, deploy, and use sensitive information in your applications

Interview

  • Introductions
  • How did you get introduced to Python?
  • To begin with, how do you define a secret in the context of an application?
  • What are the broad categories for solutions to secrets management?
  • What are the different aspects of secrets management in the lifecycle of developing, deploying, and maintaining an application?
  • How does the scale of a project or organization impact the strategies that are reasonable for secrets management?
  • What are some of the most challenging aspects of secrets management at the different stages of usage?
    • What are some of the common reasons that secrets management strategies fail?
    • What are some of the vulnerabilities or attack vectors that development teams should be thinking about when working with credentials?
  • What are your thoughts on versioning of secrets?
  • Beyond storing and deploying sensitive information, what are some of the secondary concerns around secrets management that development teams should be thinking about?
  • How does the use of multiple environments (e.g. dev, QA, production, etc.) affect the strategies used for secrets management?
  • What are some of the most useful resources that you have found for anyone looking to learn more about this subject?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Infection Monkey Vulnerability Scanner with Daniel Goldberg - Episode 177

Summary

How secure are your servers? The best way to be sure that your systems aren’t being compromised is to do it yourself. In this episode Daniel Goldberg explains how you can use his project Infection Monkey to run a scan of your infrastructure to find and fix the vulnerabilities that can be taken advantage of. He also discusses his reasons for building it in Python, how it compares to other security scanners, and how you can get involved to keep making it better.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 40Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Daniel Goldberg about Infection Monkey, an open source system breach simulation tool for evaluating the security of your network

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is infection monkey and what was the reason for building it?
    • What was the reasoning for building it in Python?
    • If you were to start over today what would you do differently?
  • Penetration testing is typically an endeavor that requires a significant amount of knowledge and experience of security practices. What have been some of the most difficult aspects of building an automated vulnerability testing system?
    • How does a deployed instance keep up to date with recent exploits and attack vectors?
  • How does Infection Monkey compare to other tools such as Nessus and Nexpose?
  • What are some examples of the types of vulnerabilities that can be discovered by Infection Monkey?
  • What kinds of information can Infection Monkey discover during a scan?
    • How does that information get reported to the user?
    • How much security experience is necessary to understand and address the findings in a given report generated from a scan?
  • What techniques do you use to ensure that the simulated compromises can be safely reverted?
  • What are some aspects of network security and system vulnerabilities that Infection Monkey is unable to detect and/or analyze?
  • For someone who is interested in using Infection Monkey what are the steps involved in getting it set up?
    • What is the workflow for running a scan?
    • Is Infection Monkey intended to be run continuously, or only with the interaction of an operator?
  • What are your plans for the future of Infection Monkey?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Continuous Delivery For Complex Systems Using Zuul with Monty Taylor - Episode 172

Summary

Continuous integration systems are important for ensuring that you don’t release broken software. Some projects can benefit from simple, standardized platforms, but as you grow or factor in additional projects the complexity of checking your deployments grows. Zuul is a deployment automation and gating system that was built to power the complexities of OpenStack so it will grow and scale with you. In this episode Monty Taylor explains how he helped start Zuul, how it is designed for scale, and how you can start using it for your continuous delivery systems. He also discusses how Zuul has evolved and the directions it will take in the future.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Join the community in the new Zulip chat workspace at podcastinit.com/chat
  • Your host as usual is Tobias Macey and today I’m interviewing Monty Taylor about Zuul, a platform that drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you start by explaining what Zuul is and how the project got started?
  • How do you view Zuul in the broader landscape of CI/CD systems (e.g. GoCD, Jenkins, Travis, etc.)?
  • What is the workflow for someone who is defining a pipeline in Zuul?
    • How are the pipelines tested and promoted?
    • One of the problems that are often encountered in CI/CD systems is the difficulty of testing changes locally. What kind of support is available in Zuul for that?
  • Can you describe the project architecture?
    • What aspects of the architecture enable it to scale to large projects and teams?
  • How difficult would it be to swap the Ansible integration for another orchestration tool?
  • What would be involved in adding support for additional version control systems?
  • What are your plans for the future of the project?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

The Pulp Artifact Repository with Bihan Zhang and Austin Macdonald - Episode 168

Summary

Hosting your own artifact repositories can have a huge impact on the reliability of your production systems. It reduces your reliance on the availability of external services during deployments and ensures that you have access to a consistent set of dependencies with known versions. Many repositories only support one type of package, thereby requiring multiple systems to be maintained, but Pulp is a platform that handles multiple content types and is easily extendable to manage everything you need for running your applications. In this episode maintainers Bihan Zhang and Austin Macdonald explain how the Pulp project works, the exciting new changes coming in version 3, and how you can get it set up to use for your deployments today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • To get worry-free releases download GoCD, the open source continous delivery server built by Thoughworks. You can use their pipeline modeling and value stream map to build, control and monitor every step from commit to deployment in one place. And with their new Kubernetes integration it’s even easier to deploy and scale your build agents. Go to podcastinit.com/gocd to learn more about their professional support services and enterprise add-ons.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • Your host as usual is Tobias Macey and today I’m interviewing Austin Macdonald and Bihan Zhang about Pulp, a platform for hosting and managing software package repositories

Interview

  • Introductions
  • How did you get introduced to Python?
  • What is Pulp and how did the project get started?
  • What are the use cases/benefits for hosting your own artifact repository?
  • What is the high level architecture of the platform?
    • Pulp 3 appears to be a fairly substantial change in architecture and design. What will be involved in migrating an existing installation to the new version when it is released?
  • What is involved in adding support for a new type of artifact/package?
  • How does Pulp compare to other artifact repositories?
  • What are the major pieces of work that are required before releasing Pulp 3?
  • What have been some of the most interesting/unexpected/challenging aspects of building and maintaining Pulp?
  • What are your plans for the future of Pulp?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Great Expectations For Your Data Pipelines with Abe Gong and James Campbell - Episode 161

Summary

Testing is a critical activity in all software projects, but one that is often neglected in data pipelines. The complexities introduced by the inherent statefulness of the problem domain and the interdependencies between systems contribute to make pipeline testing difficult to manage. To make this endeavor more manageable Abe Gong and James Campbell have created Great Expectations. In this episode they discuss how you can use the project to create tests in the exploratory phase of building a pipeline and leverage those to monitor your systems in production. They also discussed how Great Expectations works, the difficulties associated with pipeline testing and managing associated technical debt, and their future plans for the project.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Finding a bug in production is never a fun experience, especially when your users find it first. Airbrake error monitoring ensures that you will always be the first to know so you can deploy a fix before anyone is impacted. With open source agents for Python 2 and 3 it’s easy to get started, and the automatic aggregations, contextual information, and deployment tracking ensure that you don’t waste time pinpointing what went wrong. Go to podcastinit.com/airbrake today to sign up and get your first 30 days free, and 50% off 3 months of the Startup plan.
  • To get worry-free releases download GoCD, the open source continous delivery server built by Thoughworks. You can use their pipeline modeling and value stream map to build, control and monitor every step from commit to deployment in one place. And with their new Kubernetes integration it’s even easier to deploy and scale your build agents. Go to podcastinit.com/gocd to learn more about their professional support services and enterprise add-ons.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected]
  • Your host as usual is Tobias Macey and today I’m interviewing James Campbell and Abe Gong about Great Expectations, a tool for testing the data in your analytics pipelines

Interview

  • Introduction
  • How did you first get introduced to Python?
  • What is Great Expectations and what was your motivation for starting it?
  • What are some of the complexities associated with testing analytics pipelines?
    • What types of tests can be executed to ensure data integrity and accuracy?
  • What are some examples of the potential impact of pipeline debt?
  • What is Great Expectations and how does it simplify the process of building and executing pipeline tests?
  • What are some examples of the types of tests that can be built with Great Expectations?
  • For someone getting started with Great Expectations what does the workflow look like?
  • What was your reason for using Python for building it?
    • How does the choice of language benefit or hinder the contexts in which Great Expectations can be used?
  • What are some cases where Great Expectations would not be usable or useful?
  • What have been some of the most challenging aspects of building and using Great Expectations?
  • What are your hopes for Great Expectations going forward?

Contact Info

Picks

Links

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Scaling Deep Learning Using Polyaxon with Mourad Mourafiq - Episode 158

Summary

With libraries such as Tensorflow, PyTorch, scikit-learn, and MXNet being released it is easier than ever to start a deep learning project. Unfortunately, it is still difficult to manage scaling and reproduction of training for these projects. Mourad Mourafiq built Polyaxon on top of Kubernetes to address this shortcoming. In this episode he shares his reasons for starting the project, how it works, and how you can start using it today.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app you’ll need somewhere to deploy it, so check out Linode. With private networking, shared block storage, node balancers, and a 200Gbit network, all controlled by a brand new API you’ve got everything you need to scale up. Go to podcastinit.com/linode to get a $20 credit and launch a new server in under a minute.
  • Finding a bug in production is never a fun experience, especially when your users find it first. Airbrake error monitoring ensures that you will always be the first to know so you can deploy a fix before anyone is impacted. With open source agents for Python 2 and 3 it’s easy to get started, and the automatic aggregations, contextual information, and deployment tracking ensure that you don’t waste time pinpointing what went wrong. Go to podcastinit.com/airbrake today to sign up and get your first 30 days free, and 50% off 3 months of the Startup plan.
  • To get worry-free releases download GoCD, the open source continous delivery server built by Thoughworks. You can use their pipeline modeling and value stream map to build, control and monitor every step from commit to deployment in one place. And with their new Kubernetes integration it’s even easier to deploy and scale your build agents. Go to podcastinit.com/gocd to learn more about their professional support services and enterprise add-ons.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • Your host as usual is Tobias Macey and today I’m interviewing Mourad Mourafiq about Polyaxon, a platform for building, training and monitoring large scale deep learning applications.

Interview

  • Introductions
  • How did you get introduced to Python?
  • Can you give a quick overview of what Polyaxon is and your motivation for creating it?
  • What is a typical workflow for building and testing a deep learning application?
  • How is Polyaxon implemented?
    • How has the internal architecture evolved since you first started working on it?
    • What is unique to deep learning workloads that makes it necessary to have a dedicated tool for deploying them?
    • What does Polyaxon add on top of the existing functionality in Kubernetes?
  • It can be difficult to build a docker container that holds all of the necessary components for a complex application. What are some tips or best practices for creating containers to be used with Polyaxon?
  • What are the relative tradeoffs of the various deep learning frameworks that you support?
  • For someone who is getting started with Polyaxon what does the workflow look like?
    • What is involved in migrating existing projects to run on Polyaxon?
  • What have been the most challenging aspects of building Polyaxon?
  • What are your plans for the future of Polyaxon?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

Software Architecture For Developers with Neal Ford - Episode 149

Summary

Whether it is intentional or accidental, every piece of software has an existing architecture. In this episode Neal Ford discusses the role of a software architect, methods for improving the design of your projects, pitfalls to avoid, and provides some resources for continuing to learn about how to design and build successful systems.

Preface

  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
  • When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at podastinit.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app. And now you can deliver your work to your users even faster with the newly upgraded 200 GBit network in all of their datacenters.
  • If you’re tired of cobbling together your deployment pipeline then it’s time to try out GoCD, the open source continuous delivery platform built by the people at ThoughtWorks who wrote the book about it. With GoCD you get complete visibility into the life-cycle of your software from one location. To download it now go to podcatinit.com/gocd. Professional support and enterprise plugins are available for added piece of mind.
  • Visit the site to subscribe to the show, sign up for the newsletter, and read the show notes. And if you have any questions, comments, or suggestions I would love to hear them. You can reach me on Twitter at @Podcast__init__ or email [email protected])
  • To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
  • A few announcements before we start the show:
    • There is still time to register for the O’Reilly Software Architecture Conference in New York. Use the link podcastinit.com/sacon-new-york to register and save 20%
    • If you work with data or want to learn more about how the projects you have heard about on the show get used in the real world then join me at the Open Data Science Conference in Boston from May 1st through the 4th. It has become one of the largest events for data scientists, data engineers, and data driven businesses to get together and learn how to be more effective. To save 60% off your tickets go to podcastinit.com/odsc-east-2018 and register.
  • With many thanks to O’Reilly Media, I have two items to give away. To sign up you just need to subscribe to the mailing list at podcastinit.com and you will have the chance to win either a copy of Neal’s book, Building Evolutionary Architectures, or a Bronze ticket to the O’Reilly Software Architecture Conference in New York. I will be picking the winners on February 21st.
  • Your host as usual is Tobias Macey and today I’m interviewing Neal Ford about principles of software architecture for developers

Interview

  • Introductions
  • How did you get introduced to Python?
  • A majority of your work has been focused on software architectures and how that can be used to facilitate delivery of working systems. Can you start by giving a high level description of what software architecture is and how it fits into the overall development process?
  • One of the difficulties that arise in long-lived projects is that technical debt accrues to the point that forward progress stagnates due to fear that any changes will cause the system to stop functioning. What are some methods that developers can use to either guard against that eventuality, or address it when it happens?
  • What are some of the broad categories of architectural patterns that developers should be aware of?
  • Are there aspects of the language that a system or application is being implemented in which influence the style of architecture that is commonly used?
  • What are some architectural anti-patterns that you have found to be the most commonly occurring?
  • Software is useless if there is no way to deliver it to the end user. What are some of the challenges that are most often overlooked by engineering teams and how do you solve for them?
  • Beyond the purely technological aspects, what other elements of software production and delivery are necessary for a successful architecture?
  • What resources can you recommend for someone who is interested in learning more about software architecture, whether as an individual contributor or in a full time architect role?

Keep In Touch

Picks

Links

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA