A (Mostly) Drupal Blog

I had my ear pierced today. Ow! All right, I concede that on a scale of one-to-ten, the pain measured only two, two-point-five, but that moment when the burly employee in the Claire’s smock assaulted my earlobe with a nail gun, it did hurt.

This theme is interwoven with my past…pain, that is, not burly Claire’s-smocked employees. I’ve stumbled when traversing joists in the attic, and in grabbing a rafter, came into contact with a bare light bulb and “melted” my arm; I had a horse rear up at the moment I was removing a wrap from his leg, dislocating my shoulder when his hoof introduced itself to my armpit; another dislocation—same shoulder—when landing in a handstand on the pavement after a short flight initiated by a car hitting me; and one more dislocation when colliding with another skier, both of us at high speed. I’ve also been stung by all types of flying insects possessing no sense of humor.

So I’m no stranger to various types and levels of pain; most of them resulted from performing an activity I pursued by choice. While in most of those cases, I had no idea that ultimately pain would be involved, I can’t say the same for choosing to get my ear harpooned.

I tried to find a moyel to do it, because they typically perform their work painlessly—as I recall. The first one I phoned laughed at me, and the second became incensed, mumbling something about how a wuss should be doing Joomla.

Why would someone choose a path along which pain is assured? There are several possible reasons, including fearlessness, senselessness, insanity, and belief in the destination being more important than the pain encountered along the way.

The activity of assured pain, which I want to talk about today, is the journey from being a Drupal 6 freelancer to one adept at Drupal 7 and beyond.

I studied and worked with Drupal for two years before I felt completely confident doing heavy lifting. Despite its elegance, or perhaps because of it, Drupal has a very large and complex architecture. There is sometimes only one way of doing something with Drupal, but many times there are several ways, some non-intuitive. When several ways, there are ways to do it that are better than others, and what makes one better than another can vary based on context and circumstances. See? Even a description of why it is complex is complex!

For someone newly come to Drupal 7, the learning curve is probably about the same as it would have been for Drupal 6. There is more to the architecture, and more functionality, but the architecture is more elegant and the documentation is better. There is a certain level of pain associated with absorbing it all and using it properly, but unless the learner is having a trial by fire, the pain is tolerable.

However...

Let’s look at a different situation, one that is recognizable to many. If you are a Drupal 6 developer and your clients all have Drupal 6 sites, you might be starting to sweat these days. Many installations have been waiting for the community modules to mature before considering a migration, and ‘migration’ is a key word, because there is no upgrade from Drupal 6 to Drupal 7. That maturity is here now, and people are going to migrate, because when Drupal 8 is released, it’s bye-bye to Drupal 6.

The point is that if you have been working heads down as a Drupal 6 developer, you’re already behind the curve. Learning Drupal 7 through and through when you are expert at Drupal 6 is much more difficult than coming into Drupal 7 as a newbie. I believe it’s more difficult because of the need to understand all the differences between the two. If I never had to look at another Drupal 6 site, then it would be much less of a problem, but working on both requires much more knowledge. New functionality is one thing, as are API changes, but there’s a massively large number of things that are simply (or not simply) done differently, and knowing what they are, let alone internalizing them, is painful.

Looking back at the possible reasons for undertaking an activity that assures pain, several would apply to why a Drupal 6 freelancer would want to undertake Drupal 7, especially if it has to be done on their own penny, but what is the alternative? You have been hired because of your Drupal 6 expertise. When Drupal 7 development or theming is required, you want to be the person being hired. You don’t want to be shelved because your knowledge has dead-ended. There are things listed in Figure 1 that you can do to lessen the pain in learning Drupal 7 and in advance of Drupal 8, so happy Drupaling!

No, dear readers, this isn’t a tale of a sunken initiative with a payload that threatens the rest of core, though darn, that would’ve been good! This is a tale that I hope will entertain those already moist in Drupal droplets, while informing the dry newbies among us.

It is a megillah of mishigas¹ seemingly immune to the eroding currents of entropy. It begins on a cold November day, with yours truly staring at the home page of a mature Drupal site belonging to a newly acquired client. And there it was, by all appearances innocuous, except that it stared out at me from the center of the page, like a hibiscus in a field of Dusty Miller.

OCTOBER

It was this errant text I needed to exile from the page. You know the backstory: some other developer created the site and, for whatever reason, was no longer contactable. So it was up to me to discover the provenance of the chartreuse OCTOBER.

On a raw website, it would typically be a simple exercise, but this is Drupal. The first lesson of any architecture that embraces abstraction and extensibility is that these things tend to flow downstream. It’s easy to find your way to the mouth of a river, but finding its source is something else entirely.

If you want a quick solution to a mystery, often the best place to begin is at the end. That would be the page source, which is sometimes more accurate than an inspection of the item, because browser tools often do some cleanup. Drupal has a reputation for piling on div’s like layers in a lasagna. A benefit to this is that frequently one of the div tags will contain a class name that reveals its source. In this case the div tag contained no useful information, nor did the h1 tag.

On to the browser tools and, in particular, the CSS settings, because something is making the text chartreuse, and the name of the CSS file containing the rule could provide the answer I need. But no such luck. The CSS is inline, and is not from a rule contained in a style file.

So, what next? The text could very well be coming from a module or template file. In that case, a grep (global search) of the code base could find the source, if it is a literal, like:

print ‘OCTOBER’;

It is not. Though there are almost three dozen occurrences of the string, none help me. Of course, there is also the possibility that it is coming from a request to format a date using only the month name, which would make it nigh impossible to find, as would be the case if it were coming from a call to a service from within the PHP code. There are still some possibilities to check before tossing in the towel.

If it’s not in the code, which is the assumption I’ll make for now, then likely it’s in the database, and the first stop for database content is, well, Content. This site has been around for a while, so there is a wealth of content. However, in looking at the content that is used as such on the front page, none of it is the culprit; nary a node containing ‘OCTOBER.’ There is another possibility: a piece of content contains a PHP string that results in rendering the month name. But in looking at the nodes that make up the front page, none use embedded PHP. I can dump the database, and search the dump for ‘OCTOBER,’ but I’ll save that as a last resort.

Which type of content is most prevalent on a Drupal site and most prevalent in the hot debates of the past few years? Blocks. And no, none of the blocks are the purveyor of ‘OCTOBER.’ The next question is: does the site use Panels, here, there or anywhere? Fortunately the answer is “no,” because that would take me way above my word limit.

Like any good mystery, the protagonist (that would be moi) needs an opportunity to stumble upon a clue. And if there’s anything I’m a champ at, it’s stumbling. Yet, even I see the occasional blinding flash of the obvious.

In seeking a cause, when all else fails, make the rest fail.

I turned off JavaScript, and low and behold: chartreuse OCTOBER disappeared from the sonar screen. I was happier than Spongebob in a field of krab. Some further digging into the JavaScript being loaded on the page revealed that the text was being supplied from offsite.

A simple fix.

¹ A long tale of craziness; a FORTRAN term, or perhaps Yiddish?

pan·de·mo·ni·um /ˌpændəˈmoʊniəm/  noun  1. Wild uproar or unrestrained disorder, tumult or chaos.[1]  2. The initial phase of a Drupal project that is not self-contained[2]

Project Management

Project management is the use of techniques, policies and procedures to optimize the likelihood of completing a project with the agreed scope, on-time, and within budget. Though it is, to some, akin to the Tax Code (too much overhead with little gain), done properly it can be an indispensable component of a project.

To better understand, let’s look at the lowest common-denominators of most projects:

• Scope - the collective term for the project deliverables.
• Resources - the people, equipment, facilities and services needed to achieve the deliverables (including you)
• Time - the time available to perform the work, measured in person-hours when defining effort (two people working eight hours is 16 person-hours), or elapsed time for calendar time (working eight hours and then waiting two days is 56 hours of elapsed time)
•  Quality -  objective, but can mean more than ‘really great,’ such as needing to reach a level of certification
• Budget -  the bottom line for any freelancer not doing pro bono work

These factors are linked: changing one affects the others. If the time needs to be shortened, more resources will be used, resulting in a higher cost. So, at least one of the factors needs to be variable.

The project manager is responsible for the project plan, showing the tasks required to be completed, the resources and time necessary for each, and their dependencies, which are very important for keeping the project on track. Task 2 might not be able to begin until task 1 completes, or perhaps can begin so long as task 1 has already begun. If there is only one person available for configuration and there are 20 hours-worth of configuration tasks, each will likely have to occur serially. Once laid out, a path through the tasks that cannot slip without causing the entire project to slip can be identified: the critical path.

Then the project manager needs to assess risk; the ‘red flags.’  Some examples of risk are the project sponsor not having the authority to obtain resources, manipulate their priorities, or make decisions regarding scope, the project manager not having authority over the project resources, and a project that has a hard end date but not the assured availability of key resources. A risk assessment can result in factors needing adjustment (scope, budget, etc.), and sometimes the project being scrubbed.

A Drupal project can often be a red flag akin to pandemonium.  Here’s an example:

A company has historically run a Java shop. Somehow, into the mix has come a Drupal site created by a third party, no longer available. The company wants a similar site for another brand, and wants both sites constructed same way to reduce maintenance costs. To a Drupaler it doesn’t sound so bad, but let’s look at it from a project management perspective.

Planning

In most cases, defining the tasks necessary to produce this site (as opposed to any Drupal site) will be nigh on impossible. It’s a rare day that a Drupal site comes with documentation that traces each visible element or non-UI critical function back through all the events necessary to produce it. For example, does the item on the node page come from a view, a view template, a node template, content type field settings, a custom module, a panel, Javascript, or one or more of the dozens of contributed modules? Not only will you not know, you will not be able to accurately define the time needed to know.

Development

“You want to use which approach?” It might be of little matter using the Agile approach to develop in a purely Drupal environment or in a project where you are a subcontractor delivering a remotely-developed product, but in a project where resources like IT, QA, network folks, and SME’s (subject-matter experts) will need to be plugged in, “We do a bunch of coding and see where we are at a certain point, and then do it again,” or your best correct-speak version, will likely produce a ring around the conference room of dropped jaws sans a warm and fuzzy feeling.

Deployment

How are you going to take the finished site live? If you’ve been around enough Drupal deployments you know the drill, but have you ever explained the process to a non-Drupal person? “Well, we can export these as a feature, and probably those with Strongarm. These things, however, there’s really no easy way to move, and when we do, the paths will need to be edited and recreated…”  I won’t say it’s a mess, but it’s a long, long way away from being something that can be documented as a process.

I don’t dislike Drupal, really. Cut me and I drip blue droplets in search of botsnacks. I’m simply pragmatic…some call it curmudgeonly. You cannot take an organization used to the conventional way with measures and methods that better their chances of doing them profitably, and say, “Excuse me guys, can you move over and let us do our thing?” So, what’s the answer? I have two proposals:

I call for a Drupal standard for a site blueprint. I should be able to look at the blueprint and know that this field comes from that view, while the other field comes from a hook_node_view implementation with some template hijinks.

Second, I offer up the Whitewater method. If you’ve ever been whitewater rafting, you know there are waterfalls, rapids and eddies (areas of spiraling water). In this context, waterfalls are the portions of the project that need to have the conventional project controls. Rapids represent the critical path; the fastest route to completion (hopefully with you still afloat), and eddies are the spirals of agile development sprints occurring between waterfalls.

Happy Drupaling.

The Farmisht Freelancer is a column for the (oft irreverent) discussion of all things related to being in the business of developing, building and theming Drupal sites. Farmisht is a Yiddish word with a nuanced meaning. Picture yourself waking after not enough sleep and stumbling around in search of coffee to help your eyes focus.

It is one thing to be brought into a project as a team player, where the project is managed or you are delivering a predefined piece of it. However, that is typically not the way things happen when doing work for a small business or an initial project which will result in a business launch.

The more challenging and demanding opportunities for a freelancer are those one-man top-to-tail projects: creating the whole megillah¹. Here is a brief look at the steps that could give the poor shlub² a fighting chance.

1. Initial Meeting: The client transfers his vision to you. Determine what specifically makes or breaks its success.

   Output: Management Summary: Mirror the client’s vision back to him in your own words, for validation.

2. Reference site(s): Ask the client to point to sites exemplifying functionality that works and functionality that doesn’t.

   Output: Create a spreadsheet that will contain a row for each function, and identify that function as either a launch requirement, nice to have for launch, post-launch, or unneeded.

3. Conceptual design: Using the approved spreadsheet, decide what the site will look like.

   Output: Some conceptual prototype, such as wireframes, storyboard, etc.

4. Functional design: The details behind the elements of the conceptual design; how front-end elements should work, as well as the back-end functionality, business rules, and the seemingly small details (being able to print receipts with a receipt printer, accepting input from card-swipers, syncing with third-party applications, etc).

   Output: Functional design document.

5. Architectural design: With all the functionality designed, it’s time to determine the architectural stack that will support it. What is the caching strategy? What services will be required (wget, cron, curl, configuration outside the Drupal docroot, specific versions for applications or services, etc)?

   Output: Architectural design document.

6. Database design Of course, the Drupal schema has already been done for you, but there could well be additional tables needed, indexes defined, perhaps even the use of alternative databases, like MariaDB, NoSql, etc.

   Output: Schema(s) supplemental to the core Drupal schema.

   At this point, you have everything you need to determine effort and pricing. But as the client reacts to sticker shock by removing features or altering priorities, additional iterations may be needed.

7. Artistic design It’s time to develop the design from which the page layouts, templates, CSS, and themes, will emerge. It may not be your responsibility to develop the artistic design, but merely to ensure that the design embraces the functional and conceptual designs.

   Output: Artist comps, Photoshop files, etc.

8. Hosting selection: Armed with the architectural design, it’s time to consider what hosting service will be used. What pieces are needed beyond the normal LAMP stack and Drupal? What access will be required (ssh? cPanel?) and what granularity of control is needed?

   Output: Requirements list for the client to verify with the hosting service.

9. Development: Naturally, this is where most of your time will be spent. I normally do my development locally, with a privately accessible test site for the client to use for review. I take the site offline unless I have something I want the client to review, otherwise he can get antsy not seeing any change.
10. Customer testing: As page, widgets, and features progress, it’s good to have the client review them as soon as possible. There is no easy answer as to whether one should point at the agreement when the client rejects something that was done as designed. That said, certainly successful element/feature/page tests should be a foundation for an ultimate system test.
11. Production environment readiness: It’s best to install the test environment on the production environment for readiness testing in advance. That could be difficult if the domain name cannot be used yet; not having it available could cause some things to fail. But for now, this is more a test of whether all the pieces on the architecture, and all the configuration settings for the LAMP stack (or whichever is being used), are correct and viable.

This is a greatly condensed, summarized, and simplified approach; nothing in Drupal development is ever one-size-fits-all. I hope it sparks your imagination and gives you a helpful framework with which to succeed.

¹From the Yiddish megile, the scroll from which the story of Purim is read. Used to mean a long embroiled story. And yes, the source of the name for The Magilla Gorilla Show (in one episode he says, “Such a megillah over a gorilla.”)
²Yiddish, meaning a pitiful person.

One solicitation for a gig that I recently received was for a back-end developer. It contained the list of need-to-knows that I expected: PHP, Drupal module writing, OOP, MySQL, and so on. However, there was another list below that one, containing more technologies, with which the applicant should be “bullet-proof,” including JavaScript, jQuery, Angular, CSS, Sass, HTML5, PEAR, Apache, Linux shell-scripting, and several more. Despite the job heading, this was clearly a request for a full-stack developer.

Now, I actually can write JavaScript, jQuery, CSS, and HTML5, manage my own Linux server, and configure Apache domains, but I am far from being “bullet-proof” with these technologies. It would make little sense for someone to pay me by the hour to rip my hair out over CSS and JavaScript issues rather than using someone who is adept with them. 

Is it because I’m not capable of being bullet-proof? Nope. Over the centuries I’ve written countless programs in 370 Assembler, 1130 Assembler, Mnemonic Assembler, APL, PL/1, Fortran, COBOL, Basic, SPL (the HP one), and on and on. Even Xerox 9700 form language. I was one of those people in the university comp-sci department that people hate; someone who never had to study. Don’t misunderstand. It wasn’t a matter of being better, necessarily, but that my thought process is the jelly to programming logic’s peanut butter. I have known many developers who could code circles around me, but I also have a wealth of experience combining the ability to internalize a client’s business issue and visualizing a solution for crafting it. So, if I think that it’s not simply a matter of aptitude, then what is it? 

I have a bit in common with Steve Jobs. Huh? Yeah, really. We both grew up about the same time, were exposed to early computing at a young age, had our first programming experiences via timeshare teletype terminals and early CPU’s with toggle switches, were completely seduced by how software improved and fulfilled our thought process, and we both had part of our foundation poured at Hewlett-Packard. 

He was a visionary, predicting with certainty 20 years ago that the future of retail was going to be on the Web. He was able to do this, in part, because he noticed things more than other people, often at a visceral level. One thing Steve noticed was that the difference between middling and exceptional, for most things, was 1:2 – more often just 20%. Except, that is, for software, which stands virtually alone in technology with the difference between middling and exceptional being 1:50 or more, a vast difference.

He felt software was an art, and should be considered a liberal art. One thing this illustrates is that in software, like in other forms of art, there is a spectrum in which falls builders and artisans. The difference between the two is not caliber, but the direction in which they apply themselves, often a matter of taste, neither being better than the other. 

I often use the analogy of a “pile” person versus a “file” person. I happen to be the former. I am more comfortable with stacks of paper around me than having the paper filed away, because I become paralyzed when trying to decide how to organize the files, and then I forget to look in the files. And just like piles are deep and files are wide (well, the lateral ones), so needs to be the focus of the software craftsman versus the full-stack developer.

This isn’t a value judgement nor a comparison of the worth of the work product, it’s simply an explanation of the space in which the person needs to spend their time, as contrasted with where they want their time spent. Neither breadth over depth nor vice-versa, but you have to select one or the other. Me, I enjoy understanding the big picture of the client’s need and crafting the solution. As with Steve Jobs, I prefer to focus on product rather than process, which makes my eyes glaze over (just ask anyone who watched me absorb git). 

Early on, I chose depth over breadth: a writer rather than a linguist, a Harley mechanic rather than an engineer. With that decision comes the reality of having neither the time nor the desire to develop depth in each of the technologies in the full stack.

Breakfast time. I think I’ll head out for a short stack.