Intent has a long role in the understanding of various types of crime; while it is key to establish any form in some families of crime (e.g. harassment or stalking), for violent crime it is responsible for many gradations between categories of criminal acts. I believe this role is important, and that hate crime laws are entirely consistent with our justice system and positive elements of them.
Critics of hate crime laws often (please forgive me not providing citations here; the argument I cite is almost definitional for "critics") assert that the damage is in the death, not in the why. I feel this neglects an important element to violent crime.
A death does have a major impact on people close to the dead; the investment that a family makes in a person (time, care, effort) is not small, and the damage to social fabric is harsh when someone dies unexpectedly. The critics are not off that violent death is bad. However, the level and type of intent behind the act impact society too.
One of the features of rule of law is how it civilises people; legal norms create a reasonable expectation that some acts will be met with penalty. Conversely, people feel safety when they stay within the hard limits of law and the somewhat softer limits of common sense. These things are necessary for a reasonable life. In a sufficiently large society, a (low) level of random or negligent death is to be expected. Car accidents, bad luck of various forms, malfunctioning devices, these happen at some low base rate that mirrors quirky problems with one's body like heart attacks or fatal seizures. The lack of specificity or intent in those causes has us ignore them, provided the frequency is low. Occasionally some people who lose a loved one might undertake an effort to reduce the frequency of the random (low-frequency) event further, sometimes this is warranted, sometimes it's just understandable humanisation of faceless chance and a way to work out pain. These are categorically different from death that is intentional by some means; the person who kills in a fit of rage is judged harshly, but not as much as the person who kills after careful planning. This is partly based on figuring out if a person is likely to kill again, but also partly based on how alienating the death is to society. Violence outside the law, based on some person's private interests or uncommon morals, creates reasonable fear for safety in all of society. If someone receives an insult and guts with a knife the insulter and their whole family, rule-of-law is weakened and people might reasonably worry that either the killer is putting themselves generally above the law, or that the legal norms of free speech are being damaged; the more planning that enters into this, the more concerned they should be. However, at least with these kinds of crime, the damage is shared; all of society bears the cost and will hopefully deal with the problem with harsher legal penalties depending on the degree of deliberation and intent.
How do we analyse hate crime in that light? It is violence that aims to create different expectations of safety for different groups in society. It is more harmful because it suggests extinction of parts of society, making it impossible for that community to feel safe by giving them the entire burden of fear of damage. This can happen based on any category, even to a criterion that targets the majority; the less random an attack feels, the more it weakens the safety that rule-of-law should offer.
Hate crime laws are normally not just "a higher level of specificity" than the general public; they normally target certain groups that are already facing some kind of social struggle, or at least are based on category-systems that target known reasonable fears. They still fit the intuition that the violent crimes that most reasonably weaken expectations of effective rule-of-law merit stronger punishment, and alongside that the idea that the more narrowly that burden is shared, the higher the damage is.
Intent is not everything in crime, but it is crucial to how we judge human interaction. Negligence, or just bad luck, that results in injury or death may be tragic, but it is tragic in the same way as catching a fatal illness is; it doesn't impact our expectations. Hate crimes are entirely characterised by damage to those expectations, and I believe they are an important legal characterisation for further consideration on violent crime.