Home

I’ve been trying to imagine how I would make the pitch to our town, Belmont,  (e.g., our town meeting) to increase the number of people who live here.  That is, I’d like to make the case for greater density.  I’ve been trying to get this organized for a while now, I hardly think that this will make the case all by itself, but I’d love it if people at least thought about this, maybe had better ideas.

I think the pitch has several parts — why we need more density, what the obstacles are (there are several), and how we deal with those obstacles, rather than going “oh, shucks, obstacles, guess we can’t do that density thing”. 

To begin, why.  The main reason is that skyrocketing housing prices are socially bad, and given enough time to skyrocket, they start to be bad for the quality of life in town as “normal” people find it harder and harder to live anywhere near by.  And the longer we wait to act, the harder it is to fix the problem.

I take some of my conclusions from my experience living in Silicon Valley, which we left when we felt unable to buy a home (25 years ago) and which has since become even more unaffordable.  I recently learned that our favorite bakery in Palo Alto was closing, not because of high rent like you might normally expect, but because they could no longer hire qualified bakers; none could afford to live a reasonable commuting distance from the bakery.  This could happen here, too, in time.  Another problem is the difficulty that our children will have living anywhere in the area, unless they’re lucky enough to get a very lucrative job with one of the wealthier tech companies or strike it rich at a startup.  That’s how things are in Silicon Valley now; even as a senior employee at one of those wealthy tech companies, I’d be very reluctant to commit myself to a mortgage in Silicon Valley at today’s prices.  For most young people it’s simply impossible. That could happen here, too.  It’s a little harder to imagine this happening if you haven’t seen it happen, but I have seen it happen, I think we should very much take the Silicon Valley experience as something that is possible and should be avoided.

High housing prices are also socially bad.  Inequality is up for various reasons, but a spike in housing prices makes the inequality that much worse; unless you’re very wealthy, you probably have a rotten commute because you cannot live anywhere near your job.  Long commutes are bad for everyone; they’re bad for the children of parents with rotten commutes (who don’t get to see their parents, whose parents burn some of their patience just getting to and from work), they’re bad for the health of the commuters, and they’re bad for everyone who lives near the roads that the commuters drive on.  In Belmont, Cambridge, and Boston, because we lack adequate transit options to affordable suburbs, we suffer from all that traffic.  And of course, rotten commutes in not-electric cars spew pollution and greenhouse gases (electric car market share is about 2% in the US), and rotten commutes tear up our roads and cost money for maintenance, more than is collected from gas taxes and tolls (should we raise those taxes?  Seems like we should, but that also hardly seems fair to all those people forced into long commutes by our choice to let housing prices soar by capping supply).

We can attempt to mitigate cost-of-housing problems by reserving some units for affordable housing, but the higher the price of market-rate housing, the more costly (for someone) those units will be, and we don’t build nearly enough to make a dent in demand.

What these means for Belmont is that the town changes; what once was a town with a solid middle class and even a decently-housed and employed not-so-middle class, becomes instead a town of the very wealthy and the very lucky.  Because we are so adamant about not liking teardown and relatively strict zoning, the configuration of all the houses stays very much the same, but the people living in them are completely different.  I think the people are more important than a particular configuration of houses, and I think the character of the town depends more on the people living in it than on the size and shape of the houses they live in.

There are several obstacles to greater density.  The hardest and most important obstacle is how we handle many more children in our schools.  Belmont’s motto is “town of homes” (whatever that means — town of poor tax base, perhaps?) but in practice what we’re known for, and what drives demand to live here, is the quality of our schools.  Another important problem is how we would deal with additional traffic, but I think this is much more tractable.  A third problem is simply expectations; people assume that if the zoning rules are changed, they’ll be changed in a way that shortchanges them.

Zoning

I don’t think that zoning should be treated as if it were stone tablets handed down from a mountain top, because it’s not — we tweak it every few years, and recent history suggests that if every grandfathered structure in town were replaced with something that could be built by-right, quite a few lots would go vacant, and we’d complain about the ugliness and bulk of most of the rest.   People prefer the look and size of older non-conforming structures to the new ones that conform.  The same seems to be true of Somerville and Cambridge — almost every small residential property in Somerville is grandfathered and non-conforming.  What we actually dislike is change, any change.  

Neighborhoods can change in several ways — a neighborhood can maintain the same sort of people over time (locksmiths, principals, newspapermen, bartenders, electricians) and perhaps get more dense over time, or it can maintain the same density, but as property becomes more expensive, the middle class can no longer afford to buy, and all the new residents look more and more like the most-wealthy 1%, and also tend older because almost nobody earns that much money when they are young.  Over time, as the town demographics change, town preferences and policies will change, too.  Older residents might stay, but as the reward for selling goes higher and higher, and as their children end up living far away in more affordable places, the incentives to leave eventually win.  Either way, something will change.

The original purpose of zoning was also socially dubious — “poor folk” were viewed as a detriment to neighborhoods — but that doesn’t have to be the case.  We can change our zoning to reflect social needs, like a need for more housing, or a need to let people live closer to where they work.

Traffic

One big worry about increasing density is that if the new people behave like the current people, we’ll just end up with more traffic and more competition for a fixed amount of parking.  This is a risk, but I think two things mitigate it.  

The first is that the main point of adding density is that it will make the alternatives to driving more feasible.  More people taking transit means buses can run more often; in our town, there are two buses to two parts of town, and the bus that serves the dense part of town runs every 7 minutes at rush hour, and no worse than 17 minute intervals in the middle of the day; the bus that serves the not-dense part of town runs every 20 minutes at rush hour, and up to 35 minute intervals in the middle of the day.  Recently, to improve service, a lane was reserved for the more-frequent bus in a critical portion of its route because more people were traveling that section of the road by bus than by car.  The large number of people using that bus was part of what made this reserved lane politically possible — it was not “better for the planet” or “mass transit is good for social equity” or “mass transit is safer for other people” — it was literally, “this will move more people along this road in a given amount of time”.  Similarly, for bicycles, more people trying to bike to work and bike places will generate the political will to make it easier, and it might not even impede automobile travel.  The Mass Central Rail Trail is off-road, so takes no space from cars. In quite a few cases the impediment to car travel is competition for intersection time, not competition for road lanes (this is true on the Middlesex Turnpike/Lowell Street through Lexington, this is also true for Concord Avenue from Belmont to the Fresh Pond Rotary).

That is, one of the reasons for increased density is to make not-car transportation physically and politically feasible, and to create more options.  The bus that you don’t take because 20-minute intervals are not convenient, might become the bus that you do take because 10 minute intervals are convenient.  And by creating those options, we also create the option to not care so much about traffic; if you ride a bike, car traffic jams are just something that you ride through or ride past.  If you ride the bus, there can be a bus lane so you’re not delayed, and even if you are delayed, at least you’re not driving.

The second thing that might mitigate the traffic impact of increased density is that induced demand is a two-way street — if travel on a road gets easier, more people will drive there, but if travel on a road gets harder, fewer people will drive there.  Another way of phrasing this is that traffic will be pretty awful no matter what we do (till we ban it, block it, or price it), so traffic doesn’t actually matter. If, somehow, we kicked 10% of the population out of town, fewer people from in town would be clogging the roads — but more people from out-of-town would use the newly available road capacity to get to Cambridge and Boston.  This effect is becoming larger and faster-acting as more and more people use apps like Waze to quickly seek free-flowing routes and avoid traffic.  If, on the other hand, we added 10% to our population, more people who lived here would be clogging the roads — and cut-through traffic would seek other routes, commuters would reconsider using mass transit, car pool, etc, and the total traffic increase would be somewhat blunted.  At the same time, if we cut our population by 10%, our existing bus routes might have their frequency reduced because of the reduced demand, whereas increasing population increases demands, and ultimately leads to increased frequency. The #73 arrives more often because the #73 fills up.

Creating these options to get people out of cars is one of the reasons to favor increased density near jobs and transit; cars are entirely a personal good, not a social good.  Anyone in a bus wishes the cars were out of their way.  Anyone living near a road wishes fewer cars used it because cars are noisy and create and raise dust.  Anyone walking near a road would prefer fewer cars for all those reasons, plus the difficulty of crossing a road quickly and safely, especially with children.  Anyone biking on a road definitely prefers fewer cars (especially with children).  And even someone driving on a road, if they are stuck in traffic, would prefer that enough other people (but not them) would choose not to drive right then, so that they could drive faster.  And to the extent that most cars (98%) are not electric, cars pollute somewhat and emit greenhouse gases.

Another way to look at this is that our current traffic problems are in some sense a product of our zoning; we’ve zoned in a way that makes bus transit to half the town far less effective, so few or no buses run there, thus people drive instead.  

Education 

Education is a much stickier problem.  Anyone who’s rah-rah density and wonders why those bozos in Belmont don’t do their part, read this section carefully, and imagine coming up with an answer that’s not glib.

At least in Massachusetts, K-12 education is largely funded from local property taxes, and education is relatively expensive.  For Massachusetts, in 2017, average per-pupil spending was almost $16,000, but the average single-family tax bill was well lower than that, not quite $6000 in 2019.  This may be misleading because cities have both large non-residential tax bases and also large populations that proportionally influence the average.  I will instead work with the figures for Belmont, both because they are available, and because its proximity to Cambridge and Boston make it an obvious candidate for “why not greater density?”

In Belmont, the average tax bill in FY2019 was $12,720, so double the state average — there’s little business or industry in town, so almost all of the property taxes are paid by homeowners.  The tax rate is a low percentage because land is expensive.  Education spending is below the state average, and below neighboring peers, but still $13,581.74 per student in FY2017 — that is, more than the average residential property tax bill. In FY2019 the state subsidized about 1/6 of our school spending ($9.5 million out of about $60 million). That is, households with multiple school-aged children cost the town more in school spending than they pay in property taxes.  This is mentioned every time zoning changes or affordable housing are brought up at town meeting, and both the we-hate-taxes crowd and the spend-for-schools crowds unite, because the school supporters know well that our schools run very lean and are not sure how many more students they can support. (Results above average, spending below average, we have a goose that is laying golden eggs but gosh it costs so much to feed it.)

An additional constraint is the size of the school facilities themselves; there are four K-4 elementary schools, one middle school (5-8) and a high school (9-12). The middle school was rebuilt after a fire in 1995, but the school’s size was limited by state projections of declining enrollment in future years.  These projections turned out to be incorrect (the town lobbied hard for larger but was turned down, as I was told a few years ago by a member of the school committee then) and the middle school has been using additional modular classrooms since early 2017, when middle school enrollment was 1389 students.  Middle school enrollment this school year (2018/2019) was 1489 students (from belmont_public_schools_FY20.pdf, a copy of the annually updated original), and peak enrollment (based on real estate trends and the existing elementary school pipeline) is expected to be 1621 in school year 2021/2022.  That is, middle school enrollment grew by 7.2% in the last two years.

Overall enrollment, in the nine years starting at 2018/2019, is projected to increase by 7.77%, or 0.835% per year, compounded.Screen Shot 2019 06 10 at 9 13 13 PM

This increase is not a result of greater density, but changing demographics, but it is still an increase.  The number of school-aged children is expected to grow at a moderate rate over the next decade, and until the new high school is completely finished, the schools (especially elementary and middle) are pretty much full.  The new high school is necessary because the old high school has become impractical to maintain, but it is designed to accommodate two more grades to take pressure off the middle and elementary schools — elementary schools become K-3, the middle school becomes 4-6, and the upper school becomes 7-12.  This change is not reflected in any of the projections above, and it will reduce the space crunch in all the schools.  We’re spending plenty of money to make this happen.

I’m a little puzzled about how to reconcile the usual recipe for solving “our property taxes are too high” with the need for regional housing growth to match regional job growth.  Normally, the advice for a town like Belmont is to add more commercial property — that is, add jobs, not housing, because businesses on the tax rolls are a budget win.  That’s not going to improve the regional jobs-housing imbalance.  The state’s role in this is also a little puzzling, because Belmont runs a pretty good school system for not much money.  I am certain that a lot of this comes from our history of somewhat better schools and somewhat higher property values than several neighboring towns, which leads to selection for new residents-who-are-parents focused on their children’s education and who have somewhat deep pockets, but this was also true 25 years ago, with our much-more-middle-class 25-years-ago demographics (which we’re not preserving because the price of housing is spiking).  Assuming that the state government sees best-feasible education for the largest number of kids as a public good, it seems like they’d want towns with good school systems to be able to grow and educate even more students, but that’s not current policy.  The spike in housing costs also changes who can buy into towns with better school systems; the middle class is priced out.  This seems like something the state would care about, and if the state sincerely cars, they’d say it with a wheelbarrow full of money.

I don’t see an easy way around this; if we just add housing of all kinds, we’ll strain the school capacities and the school budgets, but if we do nothing the housing scarcity problem remains.

Half-assed ideas

I’m convinced we have to do something, but maintaining a small school system while adding population is a tough problem.  I grew up someplace with regional school funding (Pinellas Country, FL) where the population grew 3.3% per year the entire time I lived there, and they had a hard time keeping up (and that population growth was disproportionately older people, not children).

If the state committed to seriously funding education with state revenues, that would help remove the fiscal reason to oppose density.  In years past that funding source has not been reliable across recessions, whereas property taxes are regarded as intrinsically reliable (without anyone noticing that we defined them that way — Prop 2.5 permits a steady growth in revenues as long as property values do not crash catastrophically).  A wealth tax, i.e., a property tax generalized to cover financial instruments and bank accounts in excess of some threshold, might help here, provided that the state government diverted an adequate fraction of this into its rainy day fund during good times. (Massachusetts is constitutionally prohibited from having income tax brackets).  Or perhaps, some sort of Georgist land value tax at the state level.  I’m not sure what would happen with a wealth tax that allowed variable rates to track a stable revenue stream,like Prop 2.5 does for real estate; in recessions, the tax rate would rise, and that might make them worse.  Or, by prying money loose from people who had accumulated it, it could blunt the effect of the recession.

We have some regional cooperation already; for special ed, LABBB (Lexington Arlington Belmont Bedford Burlington) pools resources for better services and efficiency.  The local vocational education school (Minuteman) is also supported regionally by a larger collection of towns, though Belmont is not currently a member.  If there were a way to make funding more regional, say, by sharing some of the property taxes paid by an employer/employer’s-landlord with the employee’s city or town, that would reduce suburbs’ fiscal incentives for opposing new population growth.  Simply expanding revenue collection out over a larger region would do the same.

Another possibility is to arrange for the new housing to be too small to easily accommodate families, for example studios, and single-bedroom units. Quite a few towns around here do something similar for their affordable housing; they attempt to make it retirement-sized, not family-sized, in hopes of reducing the number of children that move to town.  Assuming we did this but for not-retired people, we’d want to locate it somewhat carefully to increase the chance that these new residents would be less likely to drive to work and common errands. Three obvious choices are Waverley Square (Fitchburg commuter rail line and the very-frequent #73 bus), Belmont Center (Fitchburg commuter rail line, less-frequent #74 bus (also #75), and an easy bicycle ride to Alewife commuter rail station), and the southeast corner of town, roughly Precinct 7, which has a shorter ride on the #73, and plausible bicycle access in several directions (it is past a hill, that helps).  Someone who was not really sure that people would do without cars might favor Precinct 7 because rush hour trips into Cambridge and Boston would traverse only a small bit of Belmont.  Another option might be near Blanchard Street between the railroad tracks and Concord Avenue; that gives easy bicycle access either to Alewife or into Cambridge, and the option of either the #74, #75, or #78 buses into Cambridge.  However, some of that is low, and might be vulnerable to flooding in the future.

I have in the past thought that adding small, higher-density housing would be merely “kicking the can down the road”, because what happens when a couple living in a small apartment decides they want to start a family? Either they will want to move into something larger in town (creating demand) or decide to make do with the smaller apartment that they have (increasing the school population).  I think this still adds students to the school system, but perhaps in lower proportion to added population than if we built additional family-sized housing.

Belmont has two lower-density areas that I think are problematic for adding people.  Belmont Hill is quite low density (zoned that way), but it has no transit connections, is problematic for biking because the hill is so steep, and if someone there drives into Cambridge or Boston, they travel all the way across town.  The northeast corner of town, across Little Pond from the rest of town, has some recently-built housing, and has a plausible walk to the Alewife T station, but is otherwise disconnected from the rest of town. It’s also quite low, and if sea levels rise much in the next few decades will be correspondingly more vulnerable to floods (5 feet shown, assuming some combination of sea level rise and bad storm).

One thing we might need to also do is get over the idea that a “home” (what Belmont is a town of) is a single-family detached residence.  I know that our motto is going to generate a little bit of resistance to building something that isn’t “homes” as we usually think of them here.  We in fact have quite a few 2 and 3 family homes in precincts 3, 4, 5, and 7, especially closer to Trapelo Road and Belmont Street.  Those are homes, and larger collections of smaller apartments would also be homes, and many people would be glad to live there.

I replied to someone on Twitter with an offhand remark doubting the goodness of the bicycles-as-vehicles principle, I thought I should explain it, because it’s not really simple.

I think we make two principal mistakes, cycling-as-sport, and the emphasis on bicycles-are-vehicles in the context of “vehicular” rules that are actually more automobile-centric than general.

Cycling-as-sport results in impractical bicycles.  A practical bicycle looks much like an English 3-speed, or even more so; wide tires, upright posture, a relatively comfortable saddle, and some provision for carrying stuff.  Also, fenders to keep off the rain, a chain guard to keep your clothes clean, and perhaps a built-in light.  Modern materials make an even fatter tire possible and also make the lights more effective.

In contrast, cycling-as-sport leads to bicycles variously specialized for high speeds on tracks, high speeds in road races, good handling and traction on actual mountain sides, and sometimes, good skills for tricks and stunts.  None of these bikes will keep your clothes clean, none has a fender, and they tend to lack mounting points for racks.  Racing bikes tend to position the rider in a bent-over aerodynamic posture, which is good for speed, but less good for visibility, less good in crashes (head down and forward is not good).  Mountain bikes by default are equipped with knobby tires for traction in mud, which are merely noisy and draggy on paved roads, and with wide handlebars for improved steering leverage on rough downhill terrain, which in urban traffic are a hazard in narrow spaces.  The skinny tires on road and track bikes aren’t noisy and draggy, but they are easily caught in road imperfections, vulnerable to road imperfections, and prevent you from easily riding anywhere except on well-paved roads.  You’ll also have to reinflate those skinny tires much more often.

What this means is that if some well-meaning person hears that they should bike to work, they trot down to their bike shop, and are presented with a wall (or two) of mountain bikes and a wall (or two) of road bikes and triathlon bikes, and perhaps a few “ohyeahtheseareourcitybikes” bikes off to the side.  There’s a good chance they’ll be sold a bike that they’re not comfortable on, and that doesn’t enable them to carry the stuff that they need to carry.  

And likely they’ll see, wonder about, and perhaps be sold, some of the bicycle-specific gloves, and socks, and shoes, and pants, and jackets.  This is mostly a waste of money, and in daily use, more steps added to the bicycle ride that aren’t really necessary.  And in the end, for them, bicycle commuting turns out to be impractical, why did I take the advice of those Dirty F*cking Hippies on the internet?

The sort of bikes we ought to be selling are much more common in Northern Europe; I saw a lot of them in Sweden and Denmark.  The cargo bike that I ride is sort of an extreme version of such a bike; lots of ability to carry things, no expectation that I would carry things in a backpack (my back sometimes does not like backpacks), nice fat tires, fenders, built-in lights, and a chain guard.  Basically I just get on and go, and the bike carries my stuff, and I sit super upright and see everything around me.

The vehicular cycling mistake is harder to describe, because for now, given the roads that we’ve got, vehicular cycling is partly right; roads are public ways, bicycles are a legitimate means of travel, and anyone riding a bike has the right to use those roads.  Vehicular cycling is also right when it points out the need to think about what drivers might be expecting; if you come from an unexpected direction, don’t be too surprised if drivers aren’t looking for you.

But, unfortunately, it fails when its proponents mistake a method that works for some people (including me) for a method that works for everyone. If, for some reason, you don’t feel comfortable riding in traffic, it’s not that vehicular cycling has failed you, instead you have failed vehicular cycling.  Unsurprisingly, this did nothing for improving cycling’s ride share when vehicular cycling was the only game in town, and in practice, the safety that it actually delivered (if/when taught to, and usually rejected by, most actual cyclists on the road) was unimpressive compared to safety improvements in other countries.

Vehicular cycling is also mistaken because the existing vehicular laws were first designed for cars, which means that they’re not necessarily the best fit for bicycles, and the safety margins and safety rules in those laws are often overkill for bicycles.. The need to hammer home “bicycles are vehicles” means that these differences are ignored, because it might undercut the message.  Changes to the laws “for bicycles”, whether Idaho Stop, Copenhagen Left, or bicycles-only counterflow lanes on one-way streets encounter initial default skepticism because “wait, bicycles are vehicles, now you are making special rules?” — there is an assumption that we need the “simple” bicycles-are-vehicles treatment because exceptions are bad and confusing.  And also because bicycles-are-vehicles, rule-following by bicyclists is claimed to be of paramount importance because after all those rules are for safety, and we created those rules so vehicles (ahem, “cars”) would be safe and not run into things and each other.  This is a problem, because in fact, for others on the road if not for themselves, bicycles are far safer than cars, and cyclists can see and hear what’s around them far better than drivers can, and even when they do crash into people or property, do so with orders of magnitude less energy and destructiveness.  But because we’ve made the rules our focus, safety discussions often get derailed into arguments about who follows what rules, and lose track of silly things like how to minimize deaths and serious injuries.

Vehicular cycling also fails because the laws, designed for cars, are in some sense designed more as a framework for blame than as a safety maximizer.  In countries with lower pedestrian fatality rates, “jaywalking” is not a law.  We use that law in the US to provide an excuse for crashing into pedestrians with cars, if we really cared about safety, we would use different laws (because countries that do better use different laws, and we could just copy from them.  There’s no need for research or innovation).  When cyclists, thinking of themselves as vehicles, internalize these same laws, their tendency is to be rude to pedestrians even when there is no need whatsoever and there’s no actual safety issue. “Rude” can mean gratuitous bell-ringing, yelling, or close passes, and it’s completely unnecessary.  It takes only a little forethought to not run into a pedestrian; they are visible, slow moving, we can even communicate with them in real time if we need to coordinate our motions (I do this all the time, it’s easy).  If we accept that rule #1 is to not run into pedestrians, that changes how we bike, and we quit expecting that “jaywalking” pedestrians ought to get out of “our way” (which is a ridiculous concept for a bicycle, we’re stupidly maneuverable).

Furthermore, even if you dispute the extent that our laws are a blame-framework, they are designed around the abilities and limitations of automobiles; someone on a bicycle has several safety options that drivers do not, but these options go unmentioned in the laws because the laws were designed for cars.

It’s important to remember that we’re not infinitely capable, and we’re best at what we practice.  If our default reaction to someone breaking the rules is to follow the framework of vehicular laws, on a bicycle that means it’s not the safest reaction, and in the rare case that time is tight and the default reaction matters, we’ll do the wrong thing.  So instead of thinking about jaywalking, and how it is our job to shame the law-breaking pedestrian (or to shame the wrong-way skateboarder, etc, etc), we should think about what is safest, and what our safety practices should be, and always do those.  Usually, this is some combination of modifying our path so that it passes behind the pedestrian, and so that there is so much clearance that there’s no need to alert them.  It means, if there are several dogs or small children around, that we should slow down, because we cannot possibly keep track of three or more randomly moving objects, and it would be a disaster to hit a child.  It means, in practice, to never ride between a dog and its owner, because (1) it’s common for dogs to turn back towards their owner and (2) invisible leashes are a thing.  In all these cases we can invent rules that other people should be following, or we can directly act to reduce risk.  Vehicular cycling’s emphasis on car-oriented rules leads us towards blame first, risk-reduction second, and when it really matters, that means we don’t do the best possible job of risk reduction.

I could say more, but this is probably enough. There are issues with vehicular cycling’s lack of safety-in-numbers, and with their blind spots regarding speed, and really anything that isn’t “biking like a car”, but, later.

More twitter tomfoolery

December 16, 2017

I wrote a Go program to install blocks from a file of Twitter IDs. It’s not on github yet because my development sandbox looks more like a development catbox, and I can’t clean it up too much because the program’s running right now and Twitter’s rate limited so it’ll be a while (at 5 blocks per second, about 10 days). Recall that my goal is to completely remove fascists and griefers from my Twitter feed, and from any conversation that I happen to be in — they shouldn’t even notice the opportunity to respond, never mind wasting anyone’s time with their crap.

At least as important as the program is the list of IDs to block.
It’s 4 million lines long, sorted from most-to-least-desirable-to-block order, so this is the only way to share it. I did some by-hand sampling, and the first 10% really are notably more obnoxious than the last 10%, so I may not run this all the way to completion.

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io/ioutil"
	"net/url"
	"os"
	"regexp"
	"strconv"
	"strings"
	"time"

	"github.com/BurntSushi/toml"
	"github.com/ChimeraCoder/anaconda"
)

var digits = regexp.MustCompile("[0-9]+")
var runtime = time.Now().Format(time.RFC3339)

type ConsumerAndAppKeysAndSecrets struct {
	ConKey, ConSecret, AppToken, AppSecret string
}

var caksFile = ".twitter/ids"

/* The .twitter/ids file contains four lines with string values
   obtained from the twitter developer api.

ConKey = "..."
ConSecret = "..."
AppToken = "..."
AppSecret = "..."

   For an App token and secret, you need to create an app here: https://apps.twitter.com/app/new
   This will then give you the option to create a consumer key and secret.
   (This useful information cribbed from
   https://stackoverflow.com/questions/1808855/getting-new-twitter-api-consumer-and-secret-keys )

*/

func main() {
	caks := &ConsumerAndAppKeysAndSecrets{}
	blob, err := ioutil.ReadFile(caksFile)
	if err != nil {
		fmt.Printf("There was an error opening or reading file %s: %v\n", caksFile, err)
		os.Exit(1)
		return
	}

	err = toml.Unmarshal(blob, caks)
	if err != nil {
		fmt.Printf("There was an error unmarshalling contents of %s: %v\n", caksFile, err)
		os.Exit(1)
		return
	}

	anaconda.SetConsumerKey(caks.ConKey)
	anaconda.SetConsumerSecret(caks.ConSecret)
	api := anaconda.NewTwitterApi(caks.AppToken, caks.AppSecret)
	fmt.Println("Credentials = ", *api.Credentials)

	a := os.Args
	if len(a)  0 && i%1000 == 0 {
			flush(users, i)
			users = users[:0]
		}
		user, err := api.BlockUserId(int64(u), url.Values{})
		if err != nil {
			errst := err.Error()
			if !strings.Contains(errst, "User not found.") {
				fmt.Printf("i=%d, u=%d, err=%v\n", i, u, err)
				flush(users, i)
				os.Exit(1)
			}
			fmt.Print("X")
		} else {
			fmt.Print(".")
		}
		users = append(users, user)
	}

	flush(users, len(wlids))

}

type tomlWantsStruct struct {
	users []anaconda.User
}

func flush(users []anaconda.User, ending int) {
	buf := new(bytes.Buffer)
	if err := json.NewEncoder(buf).Encode(users); err != nil {
		fmt.Printf("There was an error encoding users: %v\n", err)
		os.Exit(1)
	}
	fname := fmt.Sprintf("Blocked-%s-%08d", runtime, ending)
	err := ioutil.WriteFile(fname, buf.Bytes(), 0666)
	if err != nil {
		fmt.Printf("There was an error writing %v: %v\n", fname, err)
		os.Exit(1)
	}
}

func readFileAsUint64s(filename string) (uids []uint64, err error) {
	var b []byte
	b, err = ioutil.ReadFile(filename)
	if err != nil {
		return
	}
	bids := bytes.Split(b, []byte("\n"))
	uids = make([]uint64, 0)
	for i, bid := range bids {
		s := digits.FindString(string(bid))
		if s == "" {
			continue
		}
		var uid uint64
		uid, err = strconv.ParseUint(s, 10, 64)
		if err != nil {
			fmt.Printf("Failure to parse line %d of %s\n", i, filename)
			return
		}
		uids = append(uids, uid)
	}
	return
}

New Twitter Algorithms

November 5, 2017

My Twitter block list got unmanageably large, and blocktogether.org was not even able to remove blocks at any sort of a reasonable rate to help me fix it. So, I used my employer’s mighty-fine search engine to look for any Go packages for the Twitter API, and found Anaconda.
Read the rest of this entry »

The last time I did this, I had figures through 2011.
Now I have 2012, 2014 and 2105 (2013 seems to be missing).
Now in a Google spreadsheet, so you can look at the numbers directly and poke at the links if you want to see where the numbers came from.

In words — since 2009, each gallon of gasoline or diesel is taxed between 40 and 50 cents too low even if the only purpose of that tax is to pay for road construction and maintenance. Any other taxes (carbon, pollution, noise, congestion, health care) would be on top of that. This also does not include the maintenance or construction that we ought to be doing; this is just what is spent.

Totaled over all the fuel sold, each year since 2009 the annual shortfall totals somewhere between 75 and 100 billion dollars.

Read the rest of this entry »

Twitter algorithms

July 2, 2017

These are my rules for making Twitter more useful.

My goal, on Twitter, is a combination of finding fun and interesting stuff and to expose myself to (certain) other points of view. At work we have training on bias, unconscious and otherwise, and on techniques for reducing it and countering it. One of the instructors mentioned that you can’t just wish unconscious bias away; apparently repeated exposure to normalizing examples is required, but it takes time (this is yet another disturbing/annoying way that our brains resemble neural nets for machine learning; in this light, unconscious bias is just the result of a lifelong biased training set.)

Read the rest of this entry »

It’s a little depressing to look at how many hard it is to get all the different factions of the Democratic Party excited about helping each other. I wonder a bit if this is a case of scarcity pushing people towards fighting over scraps, and I wonder how much this is a case of Russians/Republicans using the internet to sow left-wing dissent.

At minimum, people ought to accept that each others’ problems are worthy. Is there really any question that blacks get a raw deal in this country? Or that people who are openly gay or trans are discriminated against? Or that women don’t get promotions and pay commensurate with their skills, productivity, etc? Or that unions are necessary in order to give workers an equal footing in negotiations over pay, hours, benefits, and worker safety? Or that many forms of pollution lead to statistically early death? Lack of an adequate social safety net is clearly a problem, and clearly one that can be solved, because countries that are less wealthy do a better job than we do — notably, they deliver life expectancy and lower infant mortality for less money per capita. They can afford it, so can we. Climate change? It’s happening. Slowly, but steadily, and it’s going to continue for decades-to-centuries after we finally decide to take it seriously; the only question is how fast it’s changing when enough of us finally get alarmed enough to really act. Education? College is stupidly, fantastically expensive, and to the extent this is Baumol’s Cost Disease, we should just subsidize it (other poorer countries manage to do this) and to the extent that it isn’t we should drive prices down by properly supporting public universities. Etc. These are all problems, and the Republican Party is on the wrong side of all of these issues. We shouldn’t pick just one, we should not be put off because we think labor is important but we’re a little nervous about the gays, or focus only on racism to the exclusion of college costs — there’s nothing wrong with wanting it all, we can have it all, and all of us deserve to have these problems addressed. There’s no mutual incompatibility between any of these issues.

And be a little more skeptical, say, when someone on Fox News tries to tell you that anyone who’s LGBTQ is a threat to the women and children. We’ve done plenty to make life unpleasant for people who aren’t “normal”; if someone’s out of the closet and you notice them, they must feel very strongly about it to put up with the social crap, and must have been truly miserable in the closet. This has nothing to do with your children, and everything with them wanting to live happier lives. Anyone who tells you otherwise is trying to con you into being mean to other people for no reason at all; ignore them, they’re evil.

Or, similarly, that someone might trot out some bogus statistics to try to make white people nervous about “black crime”. Some of these stats are flat lies, in other cases the data has been tortured into confessing things that aren’t true. In practice, most people are non-violent, most people are law-abiding (well, except for traffic laws, which everyone breaks very often, and traffic violence is actually a big deal). Don’t take the bait, anyone trying to convince you that blacks are a Big Crime Risk is just plain evil, ignore them, change the channel, turn off the radio. They’re trying to turn you into a racist and create dissent on the left.

There are bullshit artists trying to sow doubt about health care, too. One dishonest clown keeps trying to claim that Medicaid is worse than no health care at all, because people on Medicaid (as a population) are sicker than people who aren’t, never mind that if you’re poor and sick you’re much more motivated to sign up for Medicaid than if you’re merely poor, in which case that might seem like more of a hassle than it’s worth. This is what passes for serious statistical analysis on the right; these guys are sad, lying clowns, don’t let their obvious bullshit make you doubt the worth of providing health care.

And so on. There’s probably better examples but I’m a cis het white guy 1%er descended (father’s side) from a family with strong ties to Dartmouth, clearly I’m a traitor to my gender, race, ancestors, etc, it’s a wonder I get any of this right. The main theme is to not let one left-wing cause be split from another, and anytime you catch someone trying to do that, think about why. I honestly wonder how many of the alleged “hard-core Bernie-bros” that get noticed on the internet now are actually left wing or even American; disinformation is a real thing, and sowing dissent is a standard tactic. I supported Bernie, I sent him money, I like (or liked) his politics. But when he didn’t win the nomination, we’re done, support the nominee, got to stay focused on outcomes. I have several friends who did the same. Ask yourself *why* someone on the left would now be interested in prolonging the primary contest after we lost the general election. It makes no sense; the Republicans are uniformly terrible for everything Bernie Sanders has supported over the years, the Democrats are uniformly better, and we tried plenty hard in the primaries and Bernie didn’t make the cut. If we don’t unite, all of us, we lose ground.