Thursday, February 25, 2010

The Dreaded Telco

Today is my railing against monopolies, specifically our local telco. We have 15 schools that use the telco to provide last mile and call exchanges.

At all of our schools we get a nice big bill in the mail that the local school leader approves and sends to the Central Office. The Central Office then cuts the check and sends it out. The whole process takes 2 weeks. However there is generally a 1 week lead time for the bill to arrive from the billing date, so if the bill cycle is the 25th we will generally get the bill on the 29th. When the check goes out, most of our schools are credited just within their bill cycle period and we generally dont have an issue with billing. However one of our school's payments only seems to post 2 weeks after the check is sent out.

The telco takes no responsibility for why this happens only for this school. The problem occurs when they cut phone service to the school which is a significant public safety risk. Service has been cut twice before we ever received notice that they show an overdue balance. After spending several hours discussing this dilemma with they they provided no solutions (we threw out there figuring out why it takes so long to post for this one school, giving a slighly longer grace period for this schools' payment to post, finding a way to get the bill sooner electronically etc.). So despite the fact that we pay them regularly they have no solution on how to prevent service termination. The kicker is that we always send the payment and they always are happy to cash the check.

To add insult to injury how they manage their last mile is significantly worse then their account management practices. As we only open schools in poor and underserved neighborhoods their last mile infrastructure (even in major cities) is atrocious. We have been asked to pay over $100k for them to pull service to a new school building because they have no capacity in the block. So for us to have the privilege of paying them a significant chunck of change monthly for internet and phone service we have to pay them that amount...and you know if they put in the capacity they will immediately try to sell services to everyone else in the neighborhood piggybacking our investment. Of course they are entitled to recoup money required to build out the infrastcuture but some of that should come from the sizable margin they are making on the service bills and the remainder should be a reasonable amount, without competition there is no need to make the charge reasonable.

Wednesday, February 10, 2010

SIS support

Good support is such a key component of any technology company. Transparent, responsive, and knowledgeable are key factors to good support. Unfortunately for us we dont get that from our current SIS vendor.

We went through an upgrade to a newer version of the SIS program which changed the database technology backending the solution. Since then we have not been able to perform a restore of the data to any point in time. We were able to do this on our own prior. It is fairly apparent there is a serious configuration issue in the setup of the new build, a setup we paid them to conduct. After failing to attempt a restore we opened a ticket to try and resolve the issue. We built a VM environment to conduct the testing in.

We got to a point where the tier 3 tech we were working with said it must be an issue with the VM environment (even though we received the same exact errors in production). I can understand the stance that it is potentially the VM snapshot that has exacerbated the issue, however instead of technically going through the troubleshooting process to figure out what is wrong with the backend in the VM envirornment and attempting to address this correctly we were told to just conduct the test in the production environment...

This is one of the worst companies i have worked with as far as support of their product is concerned in an enterprise environment, and this is the one with among the largest install base...

Thursday, November 5, 2009

Data Management

Because of the type of CMO our CMO was Data Management was deemed the most important bucket. We focus on on ample, intensive, traditional learning to get our students from behind the curve and close the achivement gap. Our CMO felt the best way to do this was to focus on data driven instruction and operation. However given the fact that we had an organically grown infrastructure ecosystem, limited human resources, and a grant to implement a specific Student Information Management System our options for that first summer were very limited in our approach to addressing Data Management.

Our newly hired Director of Operations and a summer fellow we had on Staff dealt with the brunt of implementing the new Student Information System for our CMO and all of our schools. On the positive side of what we had to address, technically, the SIS we were encouraged to utilize via the grant was backended by an enterprise class database and the system was recognized as one of the market leaders. The reality was neither of those factors really meant much.

I did some very brief research on the solution we were about to take on and some research on the SIS market on a whole. It became readily apparent that the market, especially for our scale (more than 1 school but not a large district), was very nacent. The solution we were implementing was also known to be problematic in working with the data, it seemed like a black hole that sucked the data in but didnt let it back out. However it was backended by a major player in the sql market (although it was only recently moved the that back end the year we started implementing) so i thought we could always build our own systems to query the data if needed.

When we started the technical provisioning for the servers we found out 2 strange things. The first being even though we were a young organization at the time and not serving too many students the requirements (at the time) were astronomical for a server to run the application, 4GB of RAM, Dual processor box with 2GHz or above processors in the slots and a minimum of 512k cache on each processor (what was listed on their office spec guide said 512MB), 200GB of free space, etc.

When i saw those specs i said to myself wow what are we going to do send our students to Mars with an SIS?

The outrageous specifications (including the typos on their official spec sheet) was compounded by the fact that they said we needed 2 servers, because our schools spanned 2 states...yeah...

Such began our long and contentious relationship with our SIS.

Friday, October 30, 2009

Infrastructure

From the title you can tell what was determined to be the most important thing to tackle. We determined that while we would not obtain appreciable competitive advantage (read not the most important thing to help our kids learn) via our infrastructure, a solid infrastructure was determined to be the core building block for our IT environment.

Because of these we decided to commoditize our infrastructure. The 3 maxims were standardization, mobility (work anywhere/anytime) and simplicity. We developed core standards across the environment for all software and hardware that we were responsible for (we took on literally everything except copy machines). We also decided that since we were commoditizing our infrastructure, and due to the fact that the organization was in its infancy and would be going through some serious growth periods, we would "single source" outsource the management and maintenance of our entire tech infrastructure end to end.

The work anywhere/anytime mandate added some complexity to the equation. We were very early adopters of RPC over HTTPS in Outlook, getting all admin staff blackberry's with a BES and the ability to tether, pervasive wireless in all schools and offices with unified credentials, and a laptop for all staff including all teachers. We also implemented an Asterisk system that was both a great blessing and a great pain point. The Asterisk and RPC over HTTPS implementations expended some of the capital that we accrued through the Website development project.

One key thing to note is that we decided due to the way our schools operate we would have 49 weeks of absolute change control lockdown. The only time we would allow for execution on technology projects would be during the 3 weeks our teachers would have off. Of course the first year that was completely unrealistic given the nascent stage of our outsourcing agreement and our large project to implement the across the board standards while opening 2 new schools and moving 1 into a brand new facility. We were well short on our time and quality metrics for our summer infrastructure projects but in the end we did complete all of our objectives. The problem is that we left alot of pain in our wake.

The outsourcer we chose was great to us in our initial growth phase, though they had their hands full.

The work required alot of heroic actions and one of the things that i stated when i was interviewed is "we don’t want heroes". If we need heroes then we did something wrong.

After a few years we felt we had outgrown our outsourcer's service capacity, and moved on to another service provider which we are currently experiencing some growing pains with but we are hopeful that they can see us to scale.

To date the biggest challenge with our Infrastructure has been management of our outsourcer and management of our carriers. I have a vent session about ILEC's in poor neighborhoods on the docket for this blog.

Thursday, October 29, 2009

The first 100 days

When i was first hired at a Charter School Management Organization (herein Our CMO), in 2005 the organization was young in its incarnation as a CMO but it already had started 3 schools. The oldest was several years old at the time. Technology at the schools grew organically.

The key projects we identified to accomplish in the first 100 days were:
1) Get a handle on ERate and submit our applications that were due in the next month
2) Redesign our website and the websites of our schools to match our new branding initiative
3) Develop a framework for how Our CMO engages with Technology

Number 1 was tough but extremely important. Our CMO is a nonprofit CMO that opens schools in poor neighborhoods with the goal to close the achievement gap for our students.
This makes us a perfect fit for ERate. However our prior access to the funds were spotty. This project was a trial by fire. I went to USAC's fairly good website and proceeded to familiarize myself with the process. I had a limited window as the first form (form 470) was due shortly after i joined. Due to our prior spotty record filling out applications (best intentions but the process is complicated) we were hit with a Selective Review Information Request for all of our applications. Thankfully due to my strict adherence to the guidelines we passed our SRIR and were eventually approved for all of the funding we requested. Unfortunately the hard part was actually getting those approved monies disbursed in a timely manner, but that is a later discussion.

Number 2 will probably go down as my most successful endeavor in my role at this organization (for good or bad). My overall technology philosophy is KIS, or Keep it Simple (most people know the acronym has an additional S however i try to exempt negativity from all that i do whenever possible). While many of the stakeholders were dreaming of a Java enabled website with rolldown menus and fancy doodads i wanted a simple HTML site that worked well in any browser but leveraged the beautiful collateral that was available to us. In the end my adherence to design philosophy while compromising in areas that did not impact design made for a successful project. For example one school wanted cycling images and quotes on its main page. We did this with a very simple Javascript (instead of going the route of using Flash or other similar tool) and isolated the request to the home page in a way that did not impact navigation or design. This overall success, with my grasp of technical knowledge that was displayed in meetings, built up some organization capital for our new IT department of 1, capital that would soon be quickly depleted.

Number 3. The next large task was to develop a framework for how the organization viewed IT and to begin to identify projects within that frame work. We broke down technology at Our CMO into 3 areas (its always 3 isnt it). Infrastructure, Data Management, Technology in the Classroom. Prioritization decisions were made to determine which of those buckets were most important to address at the time, which projects we would attempt to tackle, and the desired timeframe to tackle them in. Oh and we needed to open 2 new schools and move one other during the summer, and manage growth in every school as well...but that is for a later article...

Tuesday, November 4, 2008

My first blog

Hello world.

Have to do that in every new technology endeavor.

At the goading of several colleges, and after 12 years working in IT, and almost always having it on the radar, i have finally started my first blog.

It became apparent that i needed to record the trials, travails, and successes of what is going on in this amazing movement, the public non profit charter school movement, that i am privileged to be a part of. I will be bringing a personal perspective of why i am so invested in this movement as well as the IT perspective on how we can hope to change the IT aspect of K-12 education.

My first posts will be personal in nature. I believe to provide as transparent a perspective as possible you need to understand where the person is coming from. After these posts i will get into the meat of what we are doing on a daily basis to ensure that our students receive the finest education possible.

I hope someone finds the data in these posts useful as they take on the complex challenges that permeate this field.