A gotcha in running apps in AWS Fargate

Recently I’ve been migrating some ECS services to AWS Fargate. I have relatively smooth experience in dealing with apps in one VPC. However, I bumped into some undocumented problem when trying to deploy one app in a different VPC.

In short, I was trying to deploy my docker app into Fargate with tasks running in private subnets balanced by a ALB in public subnets. The resource in AWS is similar to the cloudformation at https://github.com/nathanpeck/aws-cloudformation-fargate/blob/master/service-stacks/private-subnet-private-loadbalancer.yml

CannotPullContainerError: error pulling image configuration: Get https://docker-images-prod.s3.amazonaws.com/registry…

I double checked all security groups, NATs, rout tables etc and have made 100% sure that Internet is accessible from within the private subnets.  However, it looked like ECS tasks are not able to pull docker images.

Wait…, it was not the image itself not being able to pull, it was just the configuration, and look at that URI, it is S3.  This clue led me to the solution.

It turned out that my VPC has got a VPC endpoint set up. But the route tables are not updated for the private subnets (that were added to the VPC later).

Once the route tables were updated with the VPC endpoint, the tasks passed the Load Balancer health check.

 

 

 

 

 

 

A command line tool for managing your photo backups

Like many, I use an external hard drive for backing up my photos. I take photo with my iphone and for important events I will use my digital cameras. Photography is not my hobby but I managed to collect about 90k files in my hard drive.

Backing up photos from difference device can be as easy as copy and pasting from your memory cards to the hard drive. With iphone, I have to use the app Photo to first import files into my laptop and them export all files to the hard drive.

The problem comes when sometimes I forgot if have or not back up photos on a memory card. To be on the safe side, I would normally copy and paste again to the hard drive just to make sure I am missing any photos. My hard drive ended up with many duplicated photo files. What making it worse is that sometimes photo files are renamed, resulting duplicated photos with different names.

Introducing Photoman

I want a tool that can help me do two things:

  1. Remove the duplicated photo files;
  2. Organise my photo files in the Year/Month directories. e.g. 2017/2017_01/my_photo_name.jpg.

I did some search but I could not really find a tool that can easily identify duplicated files, extract EXIF date data from photos and move/reorganise photos. I ended up writing my first ever Golang tool for this purpose. I’ve been using it for about one year and I would like the share it with you.

Repository: https://github.com/dlin-me/photoman

For installation and usage instruction, please visit the above Github page.

How it works

  1. photoman works on the files under the current directory ( where photoman command is executed );
  2. For photoman to work, it needs to first index all the files under the current directory. After indexing, it creates a .photoman_db files at the directory;
  3. The index it creates is a map of all files with their file paths, md5 hashes and EXIF created date;
  4. Updating the index involves making sure every file has a entry in the index, and create one if it doesn’t;
  5. It identify duplicate files by simply looking up files share same md5 hashes;
  6. Moving files is also easy by making use of the EXIF date available in the index;
  7. For files with no EXIF data, you have the option of using the file creation time instead;

Performance

  1. It indexes 90k files (280 GB) on a USB3 external hard drive in about 30 minutes ( Disk IO is the bottleneck ) ;
  2. Reindexing with no new files takes about 20 seconds;
  3. Reindexing with 2000 new files takes less than 1 minutes;
  4. De-duplicating and moving files takes less than 5 seconds;

When to not use ES6 Arrow functions

ES6 Arrow function is one of my favourite ES6 features. Its lexical this makes most sense in most cases.

 

The Problem:

 

However, I came a cross a case today where ES6 Arrow functions cause a bug when used as a JQuery callback. The code started like this:

$(document).on('blur', 'input[type=text]', function () {
  const $elem = $(this);
  if ($elem.val().length === 0) {
    // do something
  }
});

 

My ESLint config did not like the anonymous function so I changed it to arrow function:

$(document).on('blur', 'input[type=text]', () => {
  const $elem = $(this);
  if ($elem.val().length === 0) {
    // do something
  }
});

 

It looked all good and it passed all checks. But when a text input triggered a blur event, an error occurred complaining about $elem.val() is null.

I then put a breakpoint at the const line and inspected it in Chrome’s console. The $(this) still evaluated as an array of input elements (with one item) but strangely, once assigned, the $elem became something else (jquery.fn.init {}) and the $elem.val() evaluated null.

Ok, with ES6 arrow function’s Lexical This, it kind-of make sense. The lexical this in the example is actually not a Input element, it is pre-bound to the lexical this when the event handler is registered. Indeed, it is undefined if I put a console.log(this) there.  In this case, arrow function should be avoided because  lexical this is not desirable.

But how do we explain the results I had when inspecting with Chrome console ? It turned out that Chrome inspector is no good at resolving  this with the arrow function and keep reporting as if it is a normal callback function.

Conclusion:

  1. Take extra care when using ES6 arrow function when non-lexical this is desirable.
  2. Chrome browser inspector is not good at reporting Lexical This.

 

 

 

 

 

 

 

 

 

 

 

 

 

How to name props for React components

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

Today I am going to talk about one of the hard things in React.js — Naming props for your React components.

In general, prop names should share some common rules with naming variables. For example, use lower camel case (e.g. isActive) and keep the name short ( < 50 characters ). However, there are also some best practices that are specific to React component props.

Here are some of the tips I learned from my experience in working with React.

Follow prop types:

  1. Array – use plural nouns. e.g. items
  2. Number – use prefix num or postfix count, index etc that can imply a number. e.g. numItems, itemCount, itemIndex
  3. Bool – use prefix is, can, has
    • is: for visual/behavior variations. e.g. isVisible, isEnable, isActive
    • can: fore behavior variations or conditional visual variations. e.g. canToggle, canExpand, canHaveCancelButton
    • has: for toggling UI elements. e.g. hasCancelButton, hasHeader
  4. Object – use noun. e.g. item
  5. Node – use prefix node. containerNode
  6. Element – use prefix element. hoverElement

Named to descibe the component itself

Props should be named to describe the componet itself. Props describe what a component does, not why it does. A common mistake is to name props after the current user or the current environment. For example:

  • hasSubmitPermission describes the permission of the user, and the reason for variation, but not the component itself. A better name could be hasSubmitButton. i.e.
    <MyForm hasSubmitButton={user.canSubmit} />
  • isMobileScreen describes the browser window size, and the reason for variation, but not the component itself. A better name could be isCompactLayout. i.e.
    <MyForm isCompactLayout={browser.isMobileScreen} />

Another common mistake is to use a name describing children components instead of the component itself. This is more likely when the props are supposed to pass down to the children.

  • <MyList onItemClick={...} /> could be more appropriate than <MyList onClick={...} />
  • areCommentsLoading should be isLoadingComments
  • hasIcon describes the existence of a child component. If what you want is to make more room for the ‘icon’ instead of toggling an icon, consider using isSpacious instead. ‘hasIcon’ is the answer to why, not what.

Event handler props

  1. Use on prefix for prop names, e.g. onSelect, onClick
  2. Use handle prefix for handler functions, e.g. <MyComp onClick={this.handleClick} />
  3. Avoid using build-in event handler prop names for customer event. For example, use onSelect instead of onFocus or onClick if the native focus/click event is not of interest.

How composer’s stability and semantic versioning work with Git tags and branches

Today I revisited an answer I posted to stackoverflow a long time ago about composer’s stability and semantic versioning with Git tags and branches. I found myself needing to refresh my memory so I feel like posting here in my blog.

  • a Tag name is the exact version name. if you have a tag called ‘xx’, you can reference it in the package.json file as ‘xx’. if your tag follows the semantic naming convention (e.g. 1.0.1 or v1.0.1), you can refer to it using semantic syntax like ~1.0.*.
  • tags are ‘stable’ UNLESS their semantic names contains somethings like ‘RC1’, ‘RC2’, ‘alpha’ etc. (e.g. 1.0.1-rc1, 1.0.1-alpha) In those cases they can be referenced by 1.0.1@RC or 1.0.1@alpha.
  • branches are not ‘stable’ by default, numbered branch will be treated as development version. e.g. branch 2.0 will be referenced as 2.0.x-dev (note the extra .x ); Non-numbered branch name will be referenced with be prefixed with ‘dev-‘. i.e. ‘master’ branch becomes dev-master and ‘testing’ branch becomes dev-testing.
  • Branch alias is used, for example, when you want to treat the master branch as 2.0.x@dev. You might want to use the dev-master branch of package ABC but it happens that one of the packages you used in your project depends on the 2.0 branch of package ABC. As you can only use one version of package ABC in your project, you basically ask all other packages to use the dev-master when they want to use the 2.0.x@dev branch

Other issues like minimum-stability and nested dependencies are clear in the online docs and I am not repeating them here.

Avoid mixing React’s event system with native DOM event handling

As of React 0.13.3, React.js has its own event system that supports a subset of DOM events .

Your event handlers will be passed instances of SyntheticEvent, a cross-browser wrapper around the browser’s native event. It has the same interface as the browser’s native event, including stopPropagation() and preventDefault(), except the events work identically across all browsers.

If you find that you need the underlying browser event for some reason, simply use the nativeEvent attribute to get it.

What you would have expected

With the native DOM events, stopPropagation will simply stop the parent from being notified of the event. Just like in this example.

React event works in in the same way. However, you would expect reactEvent.stopPropagation() to stop not only the propagation of react event, but also the underlying native DOM event.

What it actually does

Stopping react event propagation does not stop the native DOM event propagation

Well, if I say that’s not true, you might also try reactEvent.nativeEvent.stopPropagation(). No, that does not work either.

Stopping react’s event propagation stop the propagation to the parents who are listening for react events but it does not stop the native DOM event propagating to the document object.

Here is an example.

Screen Shot 2015-09-16 at 10.39.59 pm

In this example, Grandpa wraps around Dad which in turn wraps around a Son. Dad listen the react event propagating from Son but it calls event.stopPropagation(). Grandpa thus is getting no events it is expecting. However, the top most document DOM node is still getting the click events originate from Son‘s DOM node.

Native DOM events propagate before react events are triggered and propagate

Ok, how about the other way, does stopping the native event propagation stops react’s synthetic event propagation ? The answer is yes, however, not only it stops native event propagation, it also stops all react events. Yes, all components, including Son is not getting any react events even the native DOM event is only stopped propagating at ‘Dad’ DOM node.
Here is the example code.

In this example, I used JQuery to attach click event handlers to the DOM nodes of Son, Dad and Grandpa. The handler for click event on Dad stops native event propagation by calling event.stopPropagation(). As a result, none of the React event handler has executed, not even the Son component !

This example also reveal that, react events are created and propagate after native DOM events propagating up to the top level component. When remove stopPropagation call, the order of events is as followed:
Screen Shot 2015-09-16 at 11.12.30 pm

Conclusion

React’s event system and native DOM events are isolated. It is advised to avoid mixing up native event handling and react’s event handling. Use react’s event system as much as you can.

Here is an example of smell code:

componentDidMount: function () {
    $(React.findDOMNode(this.refs.foo)).on('keyDown', function(){...});
}

Using native event handling, you will have to remember unregistering the event handler in the componentDidUnmount function. With react event, this is well taken care of.

A better way of doing this could be:

    <input type="text" onKeyDown={this.handleKeyDown} />

There are valid use cases where native event handling is required. Specially for events that react dose not support yet. For example, window resize event. Take extra care if you have to use native DOM events in your react app, and do remember unregistering the event handler in componentDidUnmount to avoid memory leak.

When an empty line makes a difference

Out team is in the process of transforming some legacy domain object PHP classes into Doctrine Entities.

We found something really interesting: With Doctrine, an empty line in a IdentifierGetter functions makes a difference. Here is the story.

Assuming you have entities Category and Product with 1:M relationships. To get the category id of a product, you will do:

$product->getCategory()->getId();

Since the database table for product supposedly has the field category_id, Doctrine has some “smart” logic ( Proxy Object ) to avoid getCategory() to trigger a database query for loading the category data. If done right, the above code will not load the category data instead, it return the id the proxy category object holds.

However, be careful with your getCategory() method ( Doctrine calls it “identifier getter”). For the above mentioned to work, your identifier getter needs to be “short”. Look at https://github.com/doctrine/common/blob/master/lib/Doctrine/Common/Proxy/ProxyGenerator.php#L847

In short your identifier getter needs to:

  1. Receive no argument
  2. start with “get”
  3. Need to match an entity field name. e.g. getCategory() matches $category.
  4. The ‘id’ field for category must match. e.g. getCategory()-&gt;getId() requires the identity field of Category entity be $id. $Id or $primaryId wont match
  5. Take no more than 4 lines.

A good example of getCategory() function looks like this:

public getCategory()
{
return $this->category;
}

It has already got 4 lines. This means: You can not add an extra line, be it comment or empty line, because it will make it more than 4 lines.

An empty line makes a difference !!

A cheat sheet for desgin patterns in JavaScript

Some time ago, I read a free book covering some of the design patterns in JavaScript. I found myself referring back to it from time to time. Here is the notes I took listed here to make a quick reference cheat sheet:

cover

The Factory Pattern

Problem to solve:

Often a class or object will contain other objects within it. The logic of creating objects lives inside the parent object and creates a dependency between the child class and parent class

Simple Factory Pattern:

Description:
Uses a separate class (often a singleton) to create instances

Example:

CarFactory::createCar(model) => Car

Complex Factory Pattern:

Description:
Uses subclasses to decide what concrete class to instantiate as a member object.

Example:

AbstractBaseFactory::createCar(model)
AWDFactory::createCar(model) => awdModelCar
SedanFactory::createCar(model) => sedanModelCar

The Bridge Pattern

Problem to solve:

decouple an abstraction from its implementation so that the two can vary independently.

Example 1: Event Handler

getBeerById(id, callback)
addEvent(element, ‘click’, function(e){ getBeerById(e.id, cb) );
instead of

addEvent(element, ‘click’, getBeerById(e) { … }); //couples the implementation with event

Example 2: Privileged methods as a bridge to gain access to the private variable

var public = function() {
var secret = 3;
this.privilegedGetter = function() {
return secret;
};
};

Example 3: Bridging Multiple Classes Together

var Class1 = function(a, b, c) {
this.a = a;
this.b = b;
this.c = c;
}

var Class2 = function(d) {
this.d = d;
};

var BridgeClass = function(a, b, c, d) {
this.one = new Class1(a, b, c);
this.two = new Class2(d);
};

The Composite Pattern

Problems to solve:

To treat a collection of objects the same as you would treat any of the particular sub-objects. It organizes sub-objects into a tree structure and allows the entire tree to be traversed.

Examples:

Form validation

Forms – >validate()
Form -> validate()
Section -> validate()
Field -> validate()

Hide/Show HTML Elements

parentDiv -> hide/show
childrenDiv -> hide/show
element -> hide/show

The Facade Pattern

Problem to solve:

The facade pattern does two things:

  1. it simplifies the interface of a class,
  2. and it decouples that class from the client code that uses it.

Example: Create a function for registering event handler in all browsers

function addEvent(el, type, fn) {
if (window.addEventListener) {
el.addEventListener(type, fn, false);
}
else if (window.attachEvent) {
el.attachEvent('on' + type, fn);
}
else {
el['on' + type] = fn;
}
}

Example: As Convenient Methods

function getFullName = function (){
return firstname + ’ ‘ + lastname;
}

The Adapter Pattern

Problem to solve:

The adapter pattern allows you to adapt existing interfaces to classes that would otherwise be incompatible.

What it is

An adapter converts one interface to another; it doesn’t remove any ability or otherwise simplify the interface.
Note that Adapter class simply wraps an object and converts the arguments given into what the function expects to receive.

The Decorator Pattern

The problem to solve:

The decorator pattern is used to transparently wrap objects within another object of the same interface. This allows you to add behavior to a method and then pass the method call on to the original object. Using decorator objects is a flexible alternative to creating subclasses.

In What Ways Can a Decorator Modify Its Component?

  1. Adding Behavior Before/After a Method
  2. Replacing a Method
  3. Adding New Methods

The Flyweight Pattern

The problem to solve:

It’s most useful in situations where large numbers of similar objects are created, causing performance problems.
The flyweight pattern is used to reduce the number of objects you need in your applications.

How

The first step is to separate the intrinsic state (information that is required by the internal methods of a class e.g. car make/model/year/) from the extrinsic state (information that can be removed from a class and stored externally e.g. owner details).
Instantiation Using a Factory ( make sure only one object is created for each intrinsic state )

Extrinsic State Encapsulated (e.g. a collection/database/composite of owners ) in a singleton Manager (e.g. car registration manager )

Examples:

  1. Tooltips and Tooltip manager
  2. Modals and Modal manager

The Proxy Pattern

A proxy is an object that can be used to control access to another object. It implements the same interface as this other object and passes on any method invocations to it.

A proxy does not modify the method calls that are passed on to the
real subject, other than to possibly add some control code.

Example:

  1. A virtual proxy controls access to a real subject that is expensive to create. It will defer instantiation of its real subject until a method is called

  2. A remote proxy is used to access an object in a different environment.

  3. A protection proxy is used to control access to certain methods based on who the client is.

The Observer Pattern

Problem to solve:

Observe the state of an object in a program and be notified when it changes.

A observable object has three methods associated with the instance:
subscribe, unSubscribe, and fire

The Command Pattern

It provides the ability to parameterize and pass around a method call, which can then be executed whenever you need it to be.

It provides the ability to parameterize and pass around a method call, which can then be executed whenever you need it to be.

Example: Undo and Logging

The Chain of Responsibility Pattern

To decouple the sender and the receiver of a request

Example: Express Middleware

Take a deeper look into context in React.js – Part 2

in my last post, I talked about what context is in react, how it works and how it is used in libraries and how it should be used in applications. In this post I will cover parent based context and owner based context in react.

If you arrived at this page from google, chances are that you have run into a warning like this:

Warning: owner-based and parent-based contexts differ ( values: xxx vs yyy ) for key ( xxx ) while mounting zzz, (see: http://fb.me/react-context-by-parent)

And if you are getting this message, chances are you are doing one of the following:

  1. You are rendering a component with a child component that defines childContextTypes and getChildContext (Example followed )
  2. trying to show a react component in a modal, tooltip, popover etc.
  3. Your getChildContext method returns some random attribute values

I don’t want to repeat what owner-based and parent-based contexts are. It has been clearly stated in the link from the warning message. React has officially said that parent-based context will be the future and owner-based context is deprecated. The warning message above is simply there to warn you that your app might not be working from version 0.14 and on.

Where components get their contexts

Understanding how a component gets its context can help explaining why parent-based and owner-based context can differ.

When react creates a component:

  1. It first look at what context the component is asking for by inspecting its contextTypes definition. If the component has no contextTypes, the context is null
  2. If the component has contextTypes, react will try to get the values for the keys in contextTypes
    • it will first find the root component by navigating up the parent tree
    • from the root component it search all ancestors for getChildContext method and use it to create context objects and get the value for the keys defined in contextTypes. It is possible that keys in closer ancestors overwrite the ones in remote ancestors
    • the result is a context value object with the same set of keys as in the contextTypes object;
    • if not all keys have found values, react will give a warning like this:

      Warning: Failed Context Types: Required context '...' was not specified in '...'. Check the render method of '...'.

    • if the component is a root/top-level component, or no ancestral components have getChildContext defined, the context is null
  3. The resulting context object will be the parent-based context of the direct child component. And will be the owner-based context of all descendant components created by the current component, i.e.those created in JSX blocks in the render method.

Why owner-based and parent-based contexts differ ?

1. Child component with childContextTypes and getChildContext

Let’s look at some code:

var Son = React.createClass({
    contextTypes: {
        bar: React.PropTypes.any
    },
    render: function () {
        return <div />;
    }
});

var Dad = React.createClass({
    childContextTypes: {
        bar: React.PropTypes.any
    },

    getChildContext: function () {
        return {
            bar: "BAR"
        };
    },

    render: function () {
        return <div>{this.props.children}</div>;
    }
});

var Grandpa = React.createClass({
    render: function () {
        return <Dad><Son /></Dad>;
    }
});

React.render(<Grandpa />, document.body);

In this example, Grandpa component is the top component, it gets a context of null. It creates Dad and Son in its render function, this means, the parent-based context for Dad is null, and the owner-based contexts for both Dad and Son are also null.

But let’s look at Son, it has declared that it need a value of bar in its context. It is nested inside Dad and Grandpa. Upon created, react will look at both Dad and Grandpa for getChildContext, and in our case, only Dad has getChildContext and the returned context object is {bar: “BAR”}. Therefore, Son has a parent-based context of {bar: “BAR”}. This is different from its owner-based context of null. React will complain about this difference by issuing a warning.

2. Moving a component away from its parent (e.g. Modal)

Another reason for parent-based and owner-based context differs is that the child component is moved away from its parent.

A component gets its owner-based component from its creator. I many cases, the owner-based context will be the same as the parent-based component because the parent and the child components are created at the same time in the grand parent component. e.g. render: function() { return <A><B /></A>; } , Component instance A and B will have the same owner-based context. And B’s parent-based context is also identical to A’s context, because they get the context from the same getChildContext methods of their common ancestors.

However, for modal, tooltip etc. the content node of the modal is often moved and appended to then end of Body node. When we move B’s dom node into the the end of the body node, B’s owner-based remains, however, its parent context becomes null. This triggers the warning of context differ.

3. getChildContext returns random or inconsistent values

If you’ve got a component with :

    // Component A
    getChildContext: function () {
        return {
            bar: Math.random()
        };
    },
    ...
    render: function(){
        return <B><C/></B>;
    }

It is almost guaranteed that B and C will have different contexts because both of them get their own context by calling A’s getChildContext method but that method returns different values every time it is called.

Now component C will find its owner-based context different from its parent-based context (B’s context). React will complain about this difference.

The solution

In the next post, I am going to cover ways to address this issue.