On the maximum field of linearity of linear sets

Let $V$ denote an $r$-dimensional $F_{q^n}$-vector space. For an $m$-dimensional $F_q$-subspace $U$ of $V$, assume that ${dim}_q\left({⟨v⟩}_{F_{q^n}},\cap ,U\right)⩾2$ for each nonzero vector $v\in U$. If $n⩽q$, then we prove the existence of an integer $1<d\mid n$ such that the set of one-dimensional $F_{q^n}$-subspaces generated by nonzero vectors of $U$ is the same as the set of one-dimensional $F_{q^n}$-subspaces generated by nonzero vectors of ${⟨U⟩}_{F_{q^d}}$. If we view $U$ as a point set of $\mathrm{AG}(r,q^n)$, it means that $U$ and ${⟨U⟩}_{F_{q^d}}$ determine the same set of directions. We prove a stronger statement when $n\mid m$. In terms of linear sets, it means that an $F_q$-linear set of $\mathrm{PG}(r-1,q^n)$ has maximum field of linearity $F_q$ only if it has a point of weight one. We also present some consequences regarding the size of a linear set.