We face today an enormous challenge of making sense of the complexity that swirls around us. It is not just data; it is the context in which stuff is happening that could change the insight we get from the data. It is not just volatility; it is the trade-off analysis that slows decisions for fear of making wrong choices. It is not a lack of tools, or models, or best practices that prevents the optimization and efficiency we crave; it is the inability to put it all together fast enough, to visualize our full range options, and our tendency to second-guess our judgment in that split second choice between door #1 or door #2, between BOLD or safe, between quantum leap or incremental change that makes all the difference.
The graphic above is a the result of visualization software used in Santa Cruz, California as part of a Santa Clara University research project to model the patterns of criminal activity and use it to help police use predictive analytics tools to improve crime prevention, inform staffing decisions, and anticipate where criminal activity might happen. No this is not 1984 or anything spooky, it is the next stage of disruptive technology applied to solve practice business and public policy problems. Get used to it, this is big!
As an architect and builder of data-driven, analytics heavy insight products and services to help energy vertical decision make make informed decisions, I continually search for the right balance between completeness and simplicity in the solutions I recommend to my clients. An end-to-end solution is all of the “stuff” that goes into taking good tools, good data, good analytics practices and methods and turn them into a fast, easy, transparent, consistent actionable options. I generally can find my customer a good solution, but the miracle of turning water into wine is in the judgment, self-confidence, and resolve the client adds to the decision process that informs, shapes, targets and refines the decision to fit the circumstance—-and the courage to say DO IT!
In talking about this business challenge with customers I have found it useful to take the jargon and “geekspeak” out and frame the business need discussion around what I call the five “V’s” problem.
The Five V’s are:
- Volume-how much data and information is involved and where are you getting it?
- Velocity-how fast is it changing?
- Veracity-do you believe the data? How frequently is it updating? Is that sufficient?
- Visualization-can you put all this stuff together, analyze it efficiently to gain actionable insight?
- Value-is the information that results useful, sufficient, timely and credible enough to make informed decisions—and how much is the outcome still dependent upon the judgment, experience and guts of the decision-maker?
Enterprise-scale business intelligence and predictive analytics solutions have helped greatly to deal with the first three “V’s” by slurping in high volumes of data and quickly organizing it in data structures that are searchable with user friendly software front-ends. These solutions often rely on a common data framework used across the enterprise, consistent analytics methods, scenario analysis or stochastic simulation techniques that help the business create a fast, efficient, repeatable, integrated basis for business analysis.
These solutions are huge improvements over previous generations or business analysis done in spreadsheets and pivot tables by people. But automating all this stuff and integrating it into enterprise systems has been hugely expensive, time consuming and makes the business unreasonably dependent upon the enterprise software vendor and system integrator for mission critical operations success. From the customers’ perspective the problem is the process can easily overwhelm and defer the value. The challenge and business pain point is that to get to V5 you had to find a way around V4.
What does this have to do with the energy industry transformation under way? The promise of smart grid, the clean energy economy, and the transformation of the energy future they present requires that the energy industry find a solution to V4 so we can achieve the goals of V5. This is the ‘where’s the beef?’ question implicit in every discussion of smart grid, renewable energy, and energy efficiency.
Smart Meter Deployment is not sufficient to get Smart Grid results. There is a tsunami of smart meter data soon to flood the industry. Instead of monthly meter readings of static start/finish consumption data, it is 15 minute pulses of data 24/7 enabling pattern analysis, isolation of uses from customer side of the meter sensors, and the meter will spin both ways as net metered customers both buy and sell energy produced to the grid at dynamic prices.
Smart grid is almost always equated with smart meter deployment. But smart meters are only one factor in getting to the promised benefits of smart grid. The other components are dynamic pricing, transmission access for renewable energy projects, and grid parity costs of new clean energy resource.
Fear of Dynamic Pricing undermines Smart Grid Customer Benefits. Most end use customers are still billed on an average cost basis for energy use. Most states are slow to implement dynamic pricing because it scares utilities, regulators and politicians who fear customers will revolt over price volatility. The “Bakersfield Effect” where PG&E got flash mobbed by angry customers at a ‘good press’ event staged for politicians as smart meters were installed in Bakersfield, California is the nightmare scenario. The customers thought those smart meters caused their bills to spike. Later analysis told us that the real culprit was the CPUC required tiered rate schedule PG&E used which priced energy consumed at progressively higher unit rates. So when hot weather provoked more air conditioning use and pushed customers into higher rate tiers they coincidentally got there spiked bills just as the smart meter was being screwed to the side of their house. They felt screwed alright but for the wrong reason.
Variations of dynamic pricing are emerging ranging from “peak day pricing” in California focused just on the hottest or coldest days of the year. Commercial and industrial customers have options for time-of-use rates, demand response to reduce load or curtailment pricing allowing the utility to shut them down on peak event days to better manage load. But residential customers hate utility bill price volatility and even where dynamic pricing is being introduced most utilities still have “budget plans” which offer level billing to avoid rate spikes. This completely defeats the policy and economic intent of dynamic pricing.
And let’s face it, introducing or imposing dynamic energy pricing in today’s sour economy is not likely to win many friends or votes even if end use customers could save a little money. And then there is the problem of the utility business model. Only a few states have “decoupled rates” so that utility earnings are not dependent upon selling more kWh of energy. But unless rates are decoupled the incentives are in conflict and no one is going to see many benefits from smart grid.
Renewable Energy Standards depend upon Unsustainable Subsidies. A goal of getting more of our energy supply from clean and renewable sources such as wind and solar is a popular. A majority of the states have adopted renewable portfolio standards (RPS) requiring utilities to procure specific percentages of their energy consumed from these renewable sources. The most common RPS target is about 20% of energy consumed but some states have higher goals such as California’s 33% goal.
The energy industry faces several business problems in achieving this renewable energy goal including the higher cost of wind and solar compared to traditional coal and natural gas fired power generation, the intermittency problems of wind and solar which are not available when the wind does not blow nor the sun shines so they must be backed up by natural gas, and transmission access has been a big problem since the best wind and sun levels are often in remote locations and require transmission lines be built to bring the energy to markets. Building transmission lines is hugely expensive and environmentally challenging with plenty of NIMBY issues.
The business challenge ahead as many states and their utilities near their RPS goals is will the state ‘declare victory’ or do as California has done and raise the target from 20% to 33%. The rate consequence of higher targets is substantial and when combined with the rate increase pressures from smart meter deployment, emissions reduction and other regulatory demand make declaring victory an increasingly attractive decision.
Grid Parity faces Global Competition and Domestic Industrial Policy. Wind and solar renewable energy have cost more than traditional coal or natural gas power supply alternatives. Those higher costs have been offset by Federal and state subsidies, tax grants, loan guarantees and other cost participation. But the size of the Federal deficit and the weakness of state revenues and budgets is calling into question the future of subsidies. Combine that political reality with the global market realities that wind turbines and solar photovoltaic (PV) panels are now being mass produced for export by China. Prices have fallen rapidly as the market is flooded with supply and demand is reduced by the impacts of the global recession. The result is China has dominant market share of the renewable energy commodities and resistance is growing from domestic producers in both the US and EU as they are pushed out of the market. Recent bankruptcies of large solar firms in the US including Evergreen Solar, SpectraWatt, and Solyndra are calling the policy of feed-in-tariffs to pay above market prices for wind and solar energy and tax subsidies to encourage development into question. Expect to hear more about “grid parity” which is the principal that wind and solar energy have now matured sufficient and the prices have dropped because of Chinese exports to compete at grid parity priced with natural gas fired generation—so do we need subsidies? This prospect of lost subsidies terrifies the wind and solar energy industry and is causing a quickening consolidation of the players. Meanwhile, competition is ruthlessly and efficiently forcing every player to get to grid parity pricing to be sustainable.
So what about V5?
V5 is getting to the value of the data, analysis and expertise to make informed energy business decisions. To put it all together requires tools and services to deal with all the complexities of V1-V3, but so far it has been tortuous to organize and visualize all this data and get to insight sufficient to make decisions. And then there is another problem facing the energy industry—more than one-half of energy industry employees including many of the experts who will use these tools to make decisions are baby boomers retiring over the next five years. How will the industry replace all this lost experience in an economy that resists hiring? You see the problem?
The energy industry like many others faces the triple threats of rapidly changing technology, highly volatile, competitive business conditions, and the systemic loss of talent and expertise when it can least afford it. It needs new solutions that help bridge the gap and enable the next generation of energy policy and business leaders to make informed, prudent decisions.
V4 is visualization software that helps us make better decisions
By leveraging the scalable analytics power of cloud-based database solutions, predictive analytics and the integrated analytics power of artificial intelligence processing visualization software increasingly will help make the energy transformation actionable. V4 is not some science fiction fantasy. V4 is one answer to help address this looming energy industry pain point.
V4 unlocks the customer benefits of smart grid by teaching machines and equipment to work together seamlessly to monitor performance, diagnose problems, begin self-healing actions and respond to emergencies faster, better, cheaper and more effectively to improve power grid security and reliability, optimize performance, reduce cost in ways we could scarcely imagine only yesterday.
V4 is visualizing and optimizing the way power grids work in real time through the practical application of high performance computing to integrate massive amounts of data from things like smart meters and dynamic pricing solutions, to automate distribution systems to self heal when a tree falls on a power line, to dispatch power plants more efficiently, to reroute power flows to reduce line losses, to call on demand response from customers volunteering to reduce their demand on the grid when called upon in exchange for price breaks (think of it as GroupOn for utility users).
V4 is harvested industry knowledge, operating history and workforce experience made actionable by enabling the thousands of workers, experts and professionals retiring over the coming decade to leave behind as their legacy the knowledge of how stuff works. This crowdsourced knowledge is the history of most companies and the collective wisdom of the industry and it is rarely written down in ways that make it useful for those who succeed them. Artificial intelligence software can today help us capture that knowledge and insight, organize it to be searchable and empower employees in the future to work smarter, better, safer as a result of the experience of their predecessors.
V4 is mobility—having the collective knowledge and expertise of the company with you 24/7. When employees in remote locations have access to the best data, the best experience, and the best analysis and are empowered to use it—companies get the best results, faster.
V4 turns speed into competitive advantage by quickly assembling, organizing, analyzing and enabling options and search engine recommendations generated by algorithms, learned and compared with experience, sorted and evaluated probabilistically and presented visually and succinctly. The result is good stuff happens, bad stuff is avoided or mitigated by the collective wisdom of the company, its employees, advisors and experts—all made actionable by person who must decide to DO IT!
The energy industry like so many others is being reinvented before our eyes by the amazing power of disruptive technology that is replacing old ways of doing business with new tools and capabilities we could scarcely imagine only yesterday. This continuous process of change empowers us. Visualization solutions offer ways to harvest the knowledge and experience of our predecessors and the best practice experience of our industry and make it actionable so we can visualize the full dimension of the business need and quickly analyze our options and trade-offs.
- A Problem For Smart Meters: People Don’t Understand Electricity (fastcompany.com)
- When Will Disruptive Technology Shift Power to Energy Customers? (insightadvisor.wordpress.com)
- Smart Grid Vendors Swim Upstream (forbes.com)
- Smart meter policies around the world (smartgridwatch.wordpress.com)
- You: Utility hopes program gives clear picture of smart-grid tech (latimes.com)