Wood

You said they don't have access to fossil fuels, but not to a forest. Using wood as fuel can be a good way to start gathering energy, in form of steam engine if you are looking for mechanical energy instead of heat. For the link:

A common hardwood, red oak, has an energy content (heat value) of 14.9 megajoules per kilogram [...]

Wood as fuel can be used in firewood, chips, wood pellets or sawdust, as residue from other process.

After your species advances more technologically, and want, for example, melt iron, they will develop charcoal using charcoal burner. From there, and there:

Charcoal burns at temperatures exceeding 1,100 degrees Celsius (2,010 degrees Fahrenheit). By comparison the melting point of iron is approximately 1,200 to 1,550 °C (2,190 to 2,820 °F). Due to its porosity, it is sensitive to the flow of air and the heat generated can be moderated by controlling the air flow to the fire. For this reason charcoal is still widely used by blacksmiths. Charcoal has been used for the production of iron since Roman times and steel in modern times where it also provided the necessary carbon. Charcoal briquettes can burn up to approximately 1,260 °C (2,300 °F) with a forced air blower forge.
[...]
Historically, charcoal was used in great quantities for smelting iron in bloomeries and later blast furnaces and finery forges.

Even more, you can make syngas with wood:

Like many other sources of carbon, charcoal can be used for the production of various syngas compositions; i.e., various CO + H₂ + CO₂ + N₂ mixtures. The syngas is typically used as fuel, including automotive propulsion, or as a chemical feedstock.

In times of scarce petroleum, automobiles and even buses have been converted to burn wood gas (a gas mixture consisting primarily of diluting atmospheric nitrogen, but also containing combustible gasses, mostly carbon monoxide) released by burning charcoal or wood in a wood gas generator. In 1931 Tang Zhongming developed an automobile powered by charcoal, and these cars were popular in China until the 1950s and in occupied France during World War II (called gazogènes).

Water

From ancients times there is a certain machine called water mill. From the link.

A watermill or water mill is a mill that uses hydropower. It is a structure that uses a water wheel or water turbine to drive a mechanical process such as milling (grinding), rolling, or hammering. Such processes are needed in the production of many material goods, including flour, lumber, paper, textiles, and many metal products. These watermills may comprise gristmills, sawmills, paper mills, textile mills, hammermills, trip hammering mills, rolling mills, wire drawing mills.

The water wheel is medieval technology, while the water turbine is current technology.

Watermill works gathering mechanical energy from a flow of water, like a river.

Wind

In addition to watermills, there exist Windmills which channels mechanical power from the wind itself. From the link:

A windmill is a mill that converts the energy of wind into rotational energy by means of vanes called sails or blades. Centuries ago, windmills usually were used to mill grain (gristmills), pump water (windpumps), or both. The majority of modern windmills take the form of wind turbines used to generate electricity, or windpumps used to pump water, either for land drainage or to extract groundwater.

The today technology is called wind turbine.

Animals

Additionally, you can use animals, primary horses, in a horsemill. From the link:

A horse mill is a mill, sometimes used in conjunction with a watermill or windmill, that uses a horse engine as the power source. Any milling process can be powered in this way, but the most frequent use of animal power in horse mills was for grinding grain and pumping water. Other animal engines for powering mills are powered by dogs, donkeys, oxen or camels. Treadwheels are engines powered by humans.

Poop! Poop! And more Poop!

But seriously, dried animal dung is a very common fuel, even today in some places.

You can burn it like wood.

Dry dung fuel (or dry manure fuel) is animal feces that has been dried
in order to be used as a fuel source. It is used as a fuel in many
countries around the world. Using dry manure as a fuel source is an
example of reuse of excreta. A disadvantage of using this kind of fuel
is increased air pollution. In India, this kind of fuel source is
known as "dung cakes". (ref)

You can power machines with it.

Stirling-Motor powered with cow dung in the Technical Collection Hochhut in Frankfurt on Main (ref)

You can power transportation.

The UK debuted its first poop-powered buses, which will transport
about 10,000 monthly comm​uters between Bath and Bristol Airport.
These "Bio-Buses" are the fruit of a partnership between the Bath Bus
Company and Bristol's sewage treatment system, which is run by a
company called GENeco. They can travel about 186 miles on the yearly
waste of five people, offering a more sustainable alternative to
natural gas-powered vehicles. (ref)

You can turn sewage into fuel in a variety of ways.

True to its rich history, poop-based energy has now evolved into a
multifaceted and diverse set of industries. In 2004, a waste
management facility in Renton, Washington received a $22,000,000
gr​ant to build a power plant that could turn sewage into electricity.
The same year, a rancher figured out how to power his dairy far​m with
cow patties and an engineering professor turned pig crap into​ crude
oil. (ref)

You can turn manure into natural gas.

Natural gas, though a significant contributor to climate change, is
the cleanest-burning fossil fuel. Turning cow manure into natural gas
would have three big advantages. First, it would turn animal waste, a
major source of carbon pollution, into a useful fuel. Second, it would
provide a new source of natural gas, which could be used to replace
dirtier fuels like coal and oil. Third, it would reduce the need for
fracking, the environmentally-destructive practice that extracts
natural gas from the earth. (ref)

Livestock waste yields biogas which is refined into natural gas (ref)

In addition to @Ender Look's answer, I propose to you,

Solar

The first patent for a solar collector was obtained by the Italian Alessandro Battaglia in Genoa, Italy, in 1886. Over the following years, invеntors such as John Ericsson and Frank Shuman developed concentrating solar-powered dеvices for irrigation, refrigеration, and locomоtion. In 1913 Shuman finished a 55 HP parabolic solar thermal energy station in Maadi, Egypt for irrigation.

I am talking about concentrated solar power to be more specific, where solar rays are concentrated to heat water which turns a turbine.

EDIT: I've seen people point out the fact that access to metals needed for large-scale solar generation wouldn't be easy. But perhaps solar would come later as industrialization opens up more options.

Wind and water

To answer your question, look no further than the old industrial regions of the US and Europe right before coal became commercially viable. Every town had a mill pond, and below it, industry. Mills, presses, machine works, you name it. Plants had central shaft drive, with belts driving individual machines. That came off a water wheel.

Wind was used to pump. The famous Aermotor windmill, for instance. They are still in business, and are fairly crabby about people asking them how to make electricity with their windmill. Theirs is made to pump.

Transportation? Canals. That is how coal first made it to market in large enough quantities to become commercially useful.

Electricity transmits

For development of electricity, lack of fossil fuels wouldn't even be a speed bump. The first electric plants of any scale were hydro - starting for instance with facilities at Niagara Falls. It is still a large piece of the energy pie, especially in places rich with it, like eastern Canada and the American South. It even shows up in dry, dry California - flow is very poor but exploitable height makes up for it, like Oroville with a paltry 2000 CFM flow but 700' of head.

Coal already had it feet planted, but if it hadn't, windmill manufacturers would have had no trouble figuring out how to design windmill blades to run the right speed and autofeather so windmills can sync onto the power grid and do useful work.

There's a real problem with wind "cutting out on you" arbitrarily. In a hydro-heavy world, you solve that by using wind-generated electricity for backpumping (pumped storage). You have a significant pound (lake, reservoir, impoundment) at the bottom and the top of a hill. When there is a wind surplus, that energy is used to pump the water from the lower pound to the upper. (Aermotors would work on a primitive windmill-at-the-hill setup). When the wind slakes, water falls down through turbines. Examples: the pumped-storage projects at Muskegon and Niagara. (At Niagara the top pound is much higher than the top of the falls).

This function adds extremely well onto existing hydro dams - just build or adapt the system to include a pound at the bottom of the dam. The pound doesn't even have to be anywhere near the dam, just at the same altitude, e.g. The Thermalito Forebay, which is the lower pound at Oroville but miles away from it. Oroville backpumps from the far-too-small Thermalito Diversion Pool at its base, but as water level drops, water flows backwards from the Forebay to refill it.

Total grid capacity is then limited by the hydro capacity + average wind. The commercial market sorts out the rest.

They will start with wood, and from that, charcoal. With access to coal comes metallurgy, better charcoal, and more metallurgy. Mechanical work can be done with aero- and hydro- mechanical power (i.e., water mills, windmills, etc.).

Then, from metallurgy and electromagnetism, comes hydroelectric power, and wind electric power. Also, thermosolar power (use of concentrator mirrors to drive a steam engine). If at some point they discover the photoelectric effect, after some time, they'll also have photovoltaic solar power.

Geothermal Vents. You could use heated water, steam, and just the hot ground for heating. You could use hot gaseous material expelled from the ground (through a vertical vent) to generate rotational power or even use the hot gas to propel something like a hot air balloon into the atmosphere for short periods of time, or have the balloon anchored or tethered to the earth (rope) with a tubular connection to the 'air' balloon from the vent to maintain its aerial advantage over other 'tribes' or for some other mechanical advantage.
Gaseous/liquid material can also be used for other things. Sulfuric in natural (or perhaps not on your world) it could be used to produce an acid based material, and then with mined metals, you could chance upon electricity via the 'electric' or lead acid battery. The acid/battery system could be contained (pottery) and available for excursions from your home.
Defense - you could use gaseous/liquid system for defense against other tribes, animals, etc. Manufacturing - the gaseous/liquid materials could be used in materials production.
Perhaps your system also produces Nitric acid, allowing/resulting for fertilizer production and greater food growing opportunities near the vents.
You would find many useful materials 'crystallizing' out of these vents as they vent into the air, water, caverns. Your civilization(s) near the vent may be more productive/successful than those trying to survive far away from them.

I could go on and on. Lots more to add - let your imagination inspire you.

Direct answer: Water (if the planet has elevated lakes or rivers) and/or the plants you mentioned (if the plants can burn)

Basically when the planet has rivers or elevated lakes the aliens could build hydroelectric power plants and generate the energy though them

Or burn plants to heat air or any other gases/fluids to turn turbines and generate electricity this way.

Peat is also used extensively in certain areas. Don't know if you want to count that as a fossil fuel or not.